A Genetic Algorithm Using Semantic
Relations for Word Sense Disambiguation
Michael Hausman
A master’s project report submitted to the Graduate Facility of the
University of Colorado at Colorado Springs
in partial fulfillment of the requirements for the degree of
Masters of Engineering in Software Engineering
Department of Computer Science
2011
This master’s project report for
Masters of Engineering in Software Engineering
degree by
Michael Hausman
has been approved for the
Department of Computer Science
By
_________________________
Jugal Kalita, Chair
_________________________
Edward Chow
_________________________
Al Brouillette
__________
Date
Hausman iii
Abstract
Word Sense Disambiguation is a formal way of saying, “Which dictionary definition is correct in
context?” Humans are adept at extracting the context of a sentence and applying it to every word. If
the sentence is “I like to swim next to the river bank,” then the word “bank” means “sloping land” and
does not mean “a financial institution.” Humans know this because the words “river” and “swim” have
very little to do with finance. Humans have years of knowledge and experience to quickly contextualize
the meaning of every word. A machine, however, has a much harder time finding the correct meaning.
It takes thousands of computations for even the simplest algorithms, which are not very accurate. Even
so, many applications such as language translators are still available and sold today. Language
translation relies heavily on word sense disambiguation. For this reason many translated sentences do
not make much sense. Solving word sense disambiguation would help with many applications such as
language translation.
This project explores solutions to the word sense disambiguation dilemma. There are a variety
of tools such as WordNet and SemCor referenced in this project. WordNet is a lexical database that
investigates several relations for comparing words and glosses of a word. SemCor is a collection of text
tagged with the proper part of speech and definitions. This researcher uses the semantic relations from
WordNet and examples from SemCor to measure which definitions are most likely to be correct in the
context of the communication.
A genetic algorithm employs these measurements to find the optimal set of definitions across
several sentences. Then, the researcher compares the algorithm to other word sense disambiguation
algorithms.
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman iv
Table of Contents
Abstract ............................................................................................................................................ iii
Table of Figures ............................................................................................................................... vii
Table of Tables................................................................................................................................ viii
Table of Equations ............................................................................................................................ ix
Table of Examples .............................................................................................................................. x
Chapter 1: Introduction .................................................................................................................... 11
Chapter 2: Background ..................................................................................................................... 15
2.1 Path-Based Approaches .................................................................................................................... 15
2.2 Information-Based Methods ............................................................................................................. 16
2.3 Gloss Based Methods ........................................................................................................................ 17
2.4 Vector Based Methods...................................................................................................................... 18
2.5 Using Multiple Approaches ............................................................................................................... 18
2.6 A Genetic Algorithm Approach ......................................................................................................... 19
2.7 Main Ideas behind the Approach in this Project............................................................................... 21
Chapter 3: Tools/Resources .............................................................................................................. 22
3.1 WordNet ........................................................................................................................................... 22
3.2 WordNet Interface ............................................................................................................................ 22
3.3 Part of Speech Tagger ....................................................................................................................... 23
3.4 SemCor .............................................................................................................................................. 24
3.5 SemEval ............................................................................................................................................. 24
3.6 OntoNotes ......................................................................................................................................... 25
Chapter 4: Introduction to Semantic Relations .................................................................................. 27
4.1 Frequency.......................................................................................................................................... 27
4.2 Hypernym.......................................................................................................................................... 29
4.3 Coordinate Sisters ............................................................................................................................. 30
4.4 Domain .............................................................................................................................................. 31
4.5 Synonym............................................................................................................................................ 32
4.6 Antonym............................................................................................................................................ 33
Chapter 5: A Word Sense Disambiguation Genetic Algorithm ............................................................ 35
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman v
5.1 Evolution of the Cost Function.......................................................................................................... 36
5.1.1 Notes on Sampling ..................................................................................................................... 37
5.1.2 Semantic Relation Investigation ................................................................................................ 39
5.1.2.1 Frequency Behavior ............................................................................................................. 41
5.1.2.2 Hypernym Behavior ............................................................................................................. 43
5.1.2.3 Coordinate Sister Behavior .................................................................................................. 45
5.1.2.4 Domain Behavior ................................................................................................................. 46
5.1.2.5 Synonym Behavior ............................................................................................................... 48
5.1.2.6 Antonym Behavior............................................................................................................... 49
5.1.3 The Optimal Cost Function ......................................................................................................... 51
5.1.4 Cost Function Method 1: Simple Addition .................................................................................. 51
5.1.5 Cost Function Method 2: Regression.......................................................................................... 52
5.1.6 Cost Function Method 3: Proportional Placement in a Range ................................................... 55
5.1.7 Cost Function Method 4: Add Sense Distribution ....................................................................... 58
5.1.8 Cost Function Method 5: Add Semantic Relation Distribution ................................................... 60
5.2 Mating ............................................................................................................................................... 62
5.2.1 Mating Method 1: Mate Top Third ............................................................................................ 63
5.2.2 Mating Method 2: Mate Middle Third ....................................................................................... 63
5.3 Mutation ........................................................................................................................................... 64
5.3.1 Mutation Function 1: Random Mutation ................................................................................... 65
5.3.2 Mutation Function 2: Semantic Relation Score Mutation .......................................................... 65
5.3.3 Mutation Function 3: Sense Distribution.................................................................................... 66
5.3.4 Mutation Function 4: Semantic Relation Distribution................................................................ 67
5.4 Main Genetic Algorithm Function ..................................................................................................... 67
5.5 Notes about Speed............................................................................................................................ 68
Chapter 6: Results and Analysis ........................................................................................................ 70
6.1 Measuring the Results ...................................................................................................................... 70
6.2 Comparison to Michael Billot............................................................................................................ 71
6.3 Comparison to Zhang ........................................................................................................................ 72
6.4 SemEval ............................................................................................................................................. 72
6.4.1 SemEval 2 ................................................................................................................................... 72
6.4.2 SemEval 3 ................................................................................................................................... 73
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman vi
6.4.3 SemEval 2007 ............................................................................................................................. 74
Chapter 7: Conclusions and Future Research ..................................................................................... 76
7.1 Algorithm Weaknesses ..................................................................................................................... 76
7.2 Future Possibilities ............................................................................................................................ 77
Appendix ......................................................................................................................................... 79
Appendix A: SemCor Files ....................................................................................................................... 79
Appendix B: Proportional Placement Statistics ...................................................................................... 92
Appendix C: Sense Distribution Statistics ............................................................................................... 94
Appendix D: Semantic Relation Distribution Statistics ........................................................................... 94
Appendix E: SemCor Results ................................................................................................................... 98
References..................................................................................................................................... 104
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman vii
Table of Figures
Figure 1: An example graph indicating the various sample areas .............................................................. 41
Figure 2: The graph of the frequency semantic relation using a noun-noun part of speech combination 42
Figure 3: The graph of the frequency semantic relation using an adjective-adjective part of speech
combination ................................................................................................................................................ 42
Figure 4: The graph of the hypernym semantic relation using a noun-noun part of speech combination 44
Figure 5: The graph of the hypernym semantic relation using a verb-verb part of speech combination .. 44
Figure 6: The graph of the coordinate sister semantic relation using a noun-noun part of speech
combination ................................................................................................................................................ 45
Figure 7: The graph of the coordinate sister semantic relation using a verb-verb part of speech
combination ................................................................................................................................................ 46
Figure 8: The graph of the domain semantic relation using a noun-noun part of speech combination.... 47
Figure 9: The graph of the domain semantic relation using an adverb-adverb part of speech combination
.................................................................................................................................................................... 47
Figure 10: The graph of the synonym semantic relation using a verb-verb part of speech combination . 48
Figure 11: The graph of the synonym semantic relation using an adjective-adjective part of speech
combination ................................................................................................................................................ 49
Figure 12: The graph of the antonym semantic relation using a noun-noun part of speech combination 50
Figure 13: The graph of the antonym semantic relation using an adjective-adjective part of speech
combination ................................................................................................................................................ 50
Figure 14: The graph indicating the shape of the ideal cost function ........................................................ 51
Figure 15: The graph indicating the shape of Cost Function Method 1 ..................................................... 52
Figure 16: A graph with an example regression equation .......................................................................... 53
Figure 17: A graph indicating an example of the regression of multiple regressions for frequency ......... 54
Figure 18: The graph indicating the shape of Cost Function Method 2 ..................................................... 55
Figure 19: A graph showing an example of the weaknesses of multiple hypernym regressions ............... 56
Figure 20: A graph showing an example using Cost Function Method 3 ................................................... 57
Figure 21: The graph indicating the shape of Cost Function Method 3 ..................................................... 58
Figure 22: The graph indicating the shape of Cost Function Method 4 ..................................................... 60
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman viii
Table of Tables
Table 1: The results from SemEval 2 ........................................................................................................... 73
Table 2: The results from SemEval 3 ........................................................................................................... 74
Table 3: The coarse results from SemEval 2007 ......................................................................................... 75
Table 4: The SemCor file letter indicating type of resource the original text came from. ......................... 79
Table 5: The various SemCor files ............................................................................................................... 79
Table 6: The values for Cost Function Method 3 used in this project ........................................................ 92
Table 7: The values for Cost Function Method 4 used in this project ........................................................ 94
Table 8: The values for Cost Function Method 5 used in this project ........................................................ 94
Table 9: The results of one run of every SemCor file using all the parts of speech.................................... 99
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman ix
Table of Equations
Equation 1 and Example 1: Three Path-Based Methods ............................................................................ 16
Equation 2 and Example 2: Three Information-Based Methods ................................................................ 17
Equation 3: Cost function for Zhang’s solution........................................................................................... 20
Equation 4: Semantic Relation Equation for Frequency ............................................................................. 28
Equation 5 and Example 4: Semantic Relation Equation and Example for Hypernyms ............................. 30
Equation 6 and Example 5: Semantic Relation Equation and Example for Coordinate Sisters .................. 31
Equation 7 and Example 6: Semantic Relation Equation and Example for Domain ................................... 31
Equation 8 and Example 7: Semantic Relation Equation and Example for Synonyms ............................... 33
Equation 9 and Example 8: Semantic Relation Equation and Example for Antonyms ............................... 34
Equation 10: The Weighted Probability Equation ...................................................................................... 38
Equation 11: The equation for Cost Function Method 2 ............................................................................ 54
Equation 12: The equation for Cost Function Method 3 ............................................................................ 57
Equation 13: The equation for sense distribution error ............................................................................. 59
Equation 14: The equation for Cost Function Method 4 ............................................................................ 59
Equation 15: The equation for semantic relation distribution error .......................................................... 61
Equation 16: The equation for Cost Function Method 5 ............................................................................ 61
Equation 17: The equation for comparing the current solution to the optimal solution........................... 66
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman x
Table of Examples
Equation 1 and Example 1: Three Path-Based Methods ............................................................................ 16
Equation 2 and Example 2: Three Information-Based Methods ................................................................ 17
Example 3: Example using the Frequency Semantic Relation Equation ..................................................... 29
Equation 5 and Example 4: Semantic Relation Equation and Example for Hypernyms ............................. 30
Equation 6 and Example 5: Semantic Relation Equation and Example for Coordinate Sisters .................. 31
Equation 7 and Example 6: Semantic Relation Equation and Example for Domain ................................... 31
Equation 8 and Example 7: Semantic Relation Equation and Example for Synonyms ............................... 33
Equation 9 and Example 8: Semantic Relation Equation and Example for Antonyms ............................... 34
Example 9: An example using the Weighted Probability Equation ............................................................ 38
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 11
Chapter 1: Introduction
This project explores word sense disambiguation. A word sense is a definition or meaning of a
word. Disambiguation serves to “remove all ambiguity.” Therefore, this research explores the technical
way of saying, “Which dictionary definition is correct in context?” For example, let us suppose that the
application is a GPS in a car that uses verbal locations. If the user says, “I want to go to the bank,” where
is the location? The noun “bank” has several meanings. Some of these possibilities are below. This
could be a financial institution to deposit a check. This could also be the river bank next to the house.
From this sentence alone, the GPS cannot tell the difference. It could just assume the most commonly
used meaning and point to the financial institution. However, if the next sentence is, “I want to go for
a swim,” then the assumption would be wrong. To understand which location is the destination, the
GPS would need to know the correct sense of the location. To understand which sense is correct, the
GPS needs to understand the context.
Possible senses of the noun “bank” (Princeton University, 2010)
1. a financial institution that accepts deposits and channels the money into lending activities
2. sloping land (especially the slope beside a body of water)
3. a supply or stock held in reserve for future use (especially in emergencies))
4. a building in which the business of banking transacted
5. an arrangement of similar objects in a row or in tiers
6. a container (usually with a slot in the top) for keeping money at home
7. a long ridge or pile
8. the funds held by a gambling house or the dealer in some gambling games
9. a slope in the turn of a road or track; the outside is higher than the inside in order to reduce
the effects of centrifugal force
10. a flight maneuver; aircraft tips laterally about its longitudinal axis (especially in turning)
Word sense disambiguation is not as simple as it sounds. In order for a machine to understand
which sense is correct, it must correlate several things. Some of these are: A) how many senses are
possible for this word? B) How do those senses correlate to other words? and C) What is important in a
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 12
sentence? In the preceding example, the machine would need to understand that swimming involves
water, that water implies sense 2, and the location of the desired body of water. If the GPS were to
accept any sentence, then the knowledge base required to connect every word would be immense. It
requires several years for humans to make sense and then making meaning of the simplest sentences.
To achieve this with a machine is significantly more complex.
The fictional GPS is not the only application of word sense disambiguation. If word sense
disambiguation were solved, then many applications would be possible. This could include computers
that actually talk and interact with humans as seen in science fiction movies. This also includes language
translators such as Google Translate (Google, 2010).
At present, many translated sentences do not make much sense or are hard to understand
because words are missing and the context is often incorrect. Just knowing the correct sense would
help convey the correct context after the translation. Other applications that benefit from word sense
disambiguation include text classification, automatic summaries, or anywhere text or language is
analyzed. This project solely focuses on the problem of word sense disambiguation.
The first step to word sense disambiguation is to understand how words relate within a specific
context. Humans understand the context of a word by looking at the surrounding words. Humans also
compare words in several different ways. For example, a human knows that a lake, a river, and the
ocean are all related because they are all bodies of water. Swimming is a water sport, so water must be
involved. Many people visit rivers that are near hills and mountains in order to enjoy the scenery.
Therefore, if the word “bank” were in the middle of the statement, then the correct sense of the word
bank must be 2. It probably is not a financial institution. Perhaps a machine could try the same
technique. The first step is to examine the surrounding words and compare them with each other. A
semantic relation allows the machine to compare two words in a specific way. For example, a hypernym
of a word is a more generic way of saying that word. A more generic way of saying “lake”, “river” and
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 13
“ocean” is “body of water.” This means that “lake”, “river”, and “ocean” all share similarity through the
hypernym semantic relation. “Hillside” and the second sense of “bank” are also similar via a hypernym
semantic relation. Semantic relations provide a way for a machine to compare words with each other.
Exploring Semantic relations is the primary lens for this project. This exploration compares
several words. Earlier, the hypernym semantic relation showed some similarity between several words.
The words “lake”, “river”, and “ocean” were similar because their hypernym was “body of water.”
However, it takes more hypernyms to compare “hillside” and “bank.” This implies that the first set is
more related to each other than the second set. It also means that a portion of the project is examining
how to properly measure each relation to account for this varying similarity. Not all words have
hypernyms. To account for this, there are several semantic relations in this project: frequency,
hypernym, coordinate sister, domain, synonym, and antonym. Each of these semantic relations has
different ways of comparing two words.
As the title suggests, semantic relations are only part of the project. A genetic algorithm uses
these semantic relations to provide the senses for a given text. A genetic algorithm uses Darwin’s
theory of evolution to evolve a solution over time for optimization problems. The advantage of a
genetic algorithm is that it does not need to compare every possible solution. This is important because
there are billions of possible sense combinations for a paragraph of text. The disadvantage of this
method is that the solution may not be the best possible combination, but it should be a “good” solution
if the genetic algorithm is performed correctly.
Chapter One of this report has introduced the concept of word sense disambiguation. Chapter
Two describes several techniques which other researches use in word sense disambiguation. Chapter
Three describes all of the tools and resources that this project uses to provide all of the sense and
semantic relation information related to this study. Chapter Four explains what the semantic relations
are and how the project measures the similarity between two words with each relation. Chapter Five
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 14
explains the genetic algorithm in this project. The major portion of this chapter explains how the
genetic algorithm transforms these measures to make word sense disambiguation an optimization
problem. Chapter Six compares the results to several researchers and competitions. Chapter Seven
concludes by suggesting possible areas for research in the future.
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 15
Chapter 2: Background
There are various ways to attempt word sense disambiguation. There are also various ways to
classify each attempt. One classification is supervised versus unsupervised. A supervised algorithm
depends on some training data to learn or compare information. The algorithm then uses that
knowledge to more accurately tag other text. An unsupervised system does not require any training
data. Typically the supervised approaches are more accurate if the training data are available.
However, this classification is extremely generic. Zesch and Gurevych classify four categories of
approaches in their paper: path-based, information content based, gloss based, and vector based (Zesch
& Gurevich, 2010). These classifications give a very high-level indication on how early researchers
approached the problem. Many of the newer researchers often combine several of these approaches to
achieve better results.
2.1 Path-Based Approaches
Path-based approaches measure the length of the path between two words. The shorter the
path, the more related the two words are. This method relies on a resource that supplies the paths.
This is usually a graph-like structure, like WordNet (Princeton University, 2010). Many early researchers
focus on a single relation, like hypernyms, and optimize for that relation. What hypernyms are is not as
important at this point as the fact that hypernyms create a tree. Once the relation is in a tree structure,
then someone can measure the paths in the tree. Rada et al. begin by measuring the hypernym edges
between the words (Rada, Mili, Bicknell, & Blettner, 1989). Then, Leacock and Chodorow normalize the
length by accounting for the tree depth (Leacock & Chodorow, 1998). Wu and Palmer also use this idea
of depth in their equation (Wu & Palmer, 1994). The difference is that Wu and Palmer use the depth to
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 16
the “lowest common subsumer,” which is where the hypernyms of two words intersect. This creates
three different measurements for the “hypernym tree” alone. An example and the two equations are
below. The equations are the main part of both Rada’s and Leacock’s methods. The majority of their
papers describe equations performance for various text examples. However the path is measured, a
path-based method takes the measurement and optimizes for the shortest path between all words.
w
Depth
w
w
w
LCS
Depth
w
w
w
w
𝑅𝑎𝑑𝑎(𝑤1 , 𝑤2 ) = 𝐿𝑒𝑛𝑔𝑡ℎ
𝑅𝑎𝑑𝑎(𝑤1 , 𝑤2 ) = 3
𝐿𝑒𝑛𝑔𝑡ℎ
𝐿𝐶(𝑤1 , 𝑤2 ) = −log
2 ∗ 𝐷𝑒𝑝𝑡ℎ
𝐿𝐶(𝑤1 , 𝑤2 ) = 0.426
2 × 𝐿𝐶𝑆 𝐷𝑒𝑝𝑡ℎ
𝑊𝑃(𝑤1 , 𝑤2 ) =
𝐿𝑒𝑛𝑔𝑡ℎ + 2 × 𝐿𝐶𝑆 𝐷𝑒𝑝𝑡ℎ
𝑊𝑃(𝑤1 , 𝑤2 ) = 0.727
Length: length of the path between the words
Depth: the longest depth to the words
W1
LCS Depth: the depth to the lowest common
subsumer
Equation 1 and Example 1: Three Path-Based
Methods
W2
Length
2.2 Information-Based Methods
Information-based methods take into account how much information the two words share. The
more information the two words share, the more similar the two words are. For example, start with the
hypernym tree structure from before. Just as before, the only important fact is that the relation creates
a tree structure. Since it is a tree structure, someone can measure the number of nodes that both
words share. The more nodes two words share, the more related the two words are. Resnik adapted
this idea and defines the “lowest common subsumer” as the point where the two words intersect in the
hypernym tree (Resnik, 1995). All the words from the top of the tree to the lowest common subsumer
are common subsumers. Resnik adds the probability of all the subsumers appearing in a corpus as the
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 17
measure of similarity. Jiang and Conrath modify Resnik’s idea to include the distance between the word
and the lowest common subsumer (Jiang & Conrath, 1997). Lin begins with Jiang’s hypothesis and uses
the universal measure from information theory instead (Lin, 1998). All three of these measures use the
same idea, which is adding the probability of a word appearing in a corpus. The difference is in the
equations. An algorithm depending on the amount of shared information indicates an informationbased approach.
wA
wB
wC
w
wLCS
w
wD
w
𝐼𝐶 (𝑤∗ ) = −log((𝑤𝑎 ) + 𝑃(𝑤𝑏 ) + ⋯ + 𝑃(𝑤∗ ))P
𝑅𝑒𝑠(𝑤1 , 𝑤2 ) = 𝐼𝐶 (𝑤𝐿𝐶𝑆 )
𝐽𝐶 (𝑤1 , 𝑤2 ) = 𝐼𝐶 (𝑤1 ) + 𝐼𝐶 (𝑤2 ) − 2 ∗ 𝐼𝐶 (𝑤𝐿𝐶𝑆 )
𝐼𝐶 (𝑤𝐿𝐶𝑆 )
𝐿𝑖𝑛(𝑤1 , 𝑤2 ) =
𝐼𝐶 (𝑤1 ) + 𝐼𝐶 (𝑤2 )
W2
W1
IC(w*): the probabilities of all the subsumers leading
up to the given word
P(w*): The probability of the given word appearing in
the selected corpus
Equation 2 and Example 2: Three Information-Based Methods
2.3 Gloss Based Methods
Gloss-based methods rely on the definition of the word. Dictionaries describe definitions with
different words. A dictionary will describe two similar words with other similar words. Keep track of all
the similar words between two senses, and the overlap becomes a measurement. Lesk is famous for
using this concept to tell a “pine cone” from an “ice cream cone” (Lesk, 1986). The words “pine” and
“cone” both have definitions that contain the words “tree” and “fruit.” This is a completely different
overlap than the senses of “ice cream” and “cone.” Simply pick the senses that provide the most
overlap with each other. There are, however, drawbacks to this method. If the glosses are not
descriptive enough, or if there are many false positives, this method will fail. Banerjee and Pederson
attempt to account for both problems with their Adapted Lesk Algorithm (Patwardhan, Banerjee, &
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 18
Pedersen, An Adapted Lesk Algorithm for Word Sense Disambiguation Using WordNet, 2002). First they
ignore any word that they deem a non-content word, such as “of” or “the.” Then they expand the
number of words to compare by looking up the glosses of words found through several semantic
relations. These semantic relations are: synonym, hypernym, hyponym, holonym, meronym, troponym,
and attribute. Gloss-based methods depend on the overlap between the glosses of two words.
2.4 Vector Based Methods
Vector-based methods take each individual measurement and represent it as a vector. The
cosine of the angle between the vectors is an indication of how related the two concepts are. If the
angle is really large, then the two measurements are not very related. If the angle is very small, then the
two measurements are related. The best example given by Zesch (Zesch & Gurevich, 2010) is from
Patwardahan et al. (Patwardhan, Using WordNet-Based Context Vectors to Estimate the Semantic
Relatedness of Concepts, 2006). They start with two words and find all of the glosses that match using
the Adapted Lesk Algorithm (Patwardhan, Banerjee, & Pedersen, An Adapted Lesk Algorithm for Word
Sense Disambiguation Using WordNet, 2002). Then they say that each word is a dimension. Each vector
is a gloss of the one of the relations from either word. The angle of the vector is the number of words
that overlap in the glosses of the Adapted Lesk Algorithm. The strength of this concept is that every
relation turns into a vector. Since every relation is a vector, it is easy to compare two words. However,
turning everything into a vector is a problematic task.
2.5 Using Multiple Approaches
Most of the preceding examples from early researchers have the same thought, especially in the
development of path-based and information-based methods. For each word, compare the word with
the surrounding words within one equation. The sense with the highest score is selected. Many of the
later researchers start to attempt using several approaches. The example cited here precedes the
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 19
Adapted Lesk Algorithm (Patwardhan, Banerjee, & Pedersen, An Adapted Lesk Algorithm for Word Sense
Disambiguation Using WordNet, 2002). This method starts with Lesk and expands on it by using several
semantic relations. One of these semantic relations, hypernyms, is the main relation in the path and
information examples. Patwardahan et al. even tried to combine Adapted Lesk with the earlier
information-based methods (Patwardhan, Banerjee, & Pedersen, Using Measures of Semantic
Relatedness for Word Sense Disambiguation, 2003). They found that the equation from Jiang and
Conrath worked best in their situation (Jiang & Conrath, 1997). Then they moved on to the vectorbased method described above, which starts with Adapted Lesk. All of these examples yield better
results with the combination of approaches.
Basile et al. realize that every part of speech has different relationships with other parts of
speech (Basile, Degemmis, Gentile, Lops, & Semeraro, 2007). A changing a noun has a different effect
on another noun than on a verb for a given relation. Also, one approach may work better on a specific
part of speech. With this in mind, Basile et al. use a different approach depending on the part of speech.
Nouns use a modified Leacock measure as a starting point (Leacock & Chodorow, 1998). Their
algorithm uses an extra Gaussian factor for the distance between the words in the disambiguation text
and a factor for the frequency in WordNet. Verbs use a similar approach, except with a different
Gaussian factor. Adjectives and Adverbs use the Adapted Lesk Algorithm (Patwardhan, Banerjee, &
Pedersen, An Adapted Lesk Algorithm for Word Sense Disambiguation Using WordNet, 2002). The
underlying concept of this application is that using a different approach for every part of speech can
improve the overall score.
2.6 A Genetic Algorithm Approach
This paper employs a genetic algorithm, and therefore it follows to investigate other proposed
genetic algorithm approaches as well. The genetic algorithm approach for word sense disambiguation
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 20
using the Zhang et al. formula is the conceptual base for this paper [15]. Zhang and fellow researchers
focus only upon nouns, but their approach can easily apply to other parts of speech.
This paper also focuses on the most important part of a genetic algorithm: the cost function.
The cost function determines how the genetic algorithm compares solutions. If a genetic algorithm
cannot compare solutions correctly, then the rest of the algorithm is irrelevant. Zhang explains the core
of the algorithm in two sentences. These two sentences prove that the approach is extremely simple
beyond the cost function. The cost function begins with the hypernym tree equation from Wu and
Palmer (Wu & Palmer, 1994). This equation is an input to an equation that accounts for the domain of a
word. The domain is the word or collection to which a word belongs. After this, the results are
weighted based on the frequency of the word. The frequency comes from statistics found in WordNet
(Princeton University, 2010). The significance of this cost function is that it relies on several semantic
relations at the same time.
𝐶𝑜𝑠𝑡 (𝑤1 , 𝑤2 )
𝑆𝑒𝑛𝑠𝑒𝐶𝑛𝑡(𝑤1 ) 𝑆𝑒𝑛𝑠𝑒𝐶𝑛𝑡(𝑤2 )
+
)
𝑇𝑜𝑡𝑎𝑙𝐶𝑛𝑡(𝑤1 ) 𝑇𝑜𝑡𝑎𝑙𝐶𝑛𝑡(𝑤2 )
1 + 𝑊𝑃(𝑤1 , 𝑤2 )
,
𝐷𝑜𝑚(𝑤1 ) = 𝐷𝑜𝑚(𝑤2 )
2
∗{
𝑊𝑃 (𝑤1 , 𝑤2 )
,
𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
2
=(
SenseCnt(w): The number of times this was referenced in WordNet corpus
TotalCnt(w): The total number of references for this word in WordNet corpus
SenseTotal(w): The total number of senses for this word
Sense(w): The sense number of the word currently in use
Dom(w*): the domain of the word given
Equation 3: Cost function for Zhang’s solution
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 21
2.7 Main Ideas behind the Approach in this Project
There are a variety of ways to solve word sense disambiguation. To combine every possible
approach is not the best option. However, this author has chosen to investigate examples from various
sources. Zhang’s idea of using several semantic relations in a genetic algorithm is very appealing
(Zhang, Zhou, & Martin, 2008). However, it may make more sense to use more semantic relations. It
also may help to examine each relation independently before combining them into a large cost function
equation. Many of those semantic relations should probably begin with equations that other
researchers have developed, such as the hypernym equation from Wu and Palmer (Wu & Palmer,
1994). It is possible that some of these relations correlate better than others, so adjusting for this may
make a more accurate solution. Basile’s idea of separating the parts of speech also seems cogent
(Basile, Degemmis, Gentile, Lops, & Semeraro, 2007). For example, there are only hypernyms of nouns
and verbs. It is for this reason that Zhang only looks at nouns in their solution. The melding of the
preceding concepts provides the starting point of this project.
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 22
Chapter 3: Tools/Resources
This project uses a variety of tools and resources that are referenced throughout the project.
The tools and resources include: WordNet, a C# interface to WordNet, a part of speech tagger from
Tokyo University, SemCor, and SemEval. Some of these tools, WordNet for example, provide all the
definitions and relations. Some resources, like SemCor, provide examples of correctly translated text.
The sections below explain what each tool/resource is and how this project uses them.
3.1 WordNet
WordNet is a publicly available lexical database developed by Princeton University (Miller). It
only defines nouns, verbs, adjectives, and adverbs. There are 206941 words across 117659 SynSets,
which are groups of synonyms, in WordNet 3.0. This means that there are 117659 unique definitions
available. Each of these SynSets relates to other SynSets in various ways, hence the “net” in WordNet.
This project uses many of these relations and definitions to disambiguate text. These relations include
antonyms, hypernyms, synonyms, meronyms, holonyms, domain, coordinate sisters, polysemy count,
and several others. All of these relations about words are why WordNet is a lexical database. WordNet
is so rich in information and so well executed that it is one of the most common tools for word sense
disambiguation.
3.2 WordNet Interface
There are a few options available in order to use WordNet. One option is to use an API for the
online version of WordNet (Princeton University, 2010). This requires constant internet access and all
the aforementioned resources to use the available version of WordNet. A second option is to use the
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 23
binary that comes with the install of WordNet. The program would have to run the executable and
parse the output with regular expressions. This is possible, but it is also very labor intensive. A third
option is to parse the data files that the WordNet executable does. There is documentation provided
with the WordNet installation explaining how those files operate. The WordNet website also provides
links to APIs that already parse the files in several different programming languages. Simply pick the
best API for the preferred language.
This project is implemented in C#. Matt Gerber of Michigan State University writes the best
WordNet API for C# (Gerber, 2010). What sets this API apart is the option to either store the WordNet
database in memory or on disk. The in-memory option is very fast, but uses ~200 MB of memory.
However, there are drawbacks to this API. There is no morphological logic to extract the base word, or
lemma, from a given word. To do this, the morph.c source file from the WordNet installation was
translated into C#. There is also no access to the polysemy file that WordNet uses. This file lists the
number of times every sense was referenced in the corpus used to create WordNet. Other than those
two flaws, the API works well.
3.3 Part of Speech Tagger
Some words occur as several parts of speech. For example, the word “blue” can be a noun,
verb, or adjective. Each of these parts of speech has several senses, or definitions. The only way to tell
the difference is to look at the context around the word. Knowing the part of speech would dramatically
decrease the number of senses that are necessary, and it simplifies the problem. In addition, many
researchers and competitions purposely focus on a single part of speech. Therefore, having an accurate
part of speech tagger is very useful in word sense disambiguation. The author has selected a part of
speech tagger from The University of Tokyo (Tsuruoka & Tsujii, 2005). This part of speech tagger is fast
(2400 tokens/sec) and very accurate (97.10% on WSJ corpus).
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 24
3.4 SemCor
Princeton University developed Semcor, which originates from the Brown Corpus (Princeton
University, 2011). The Brown Corpus is one of the original tagged collections of text for creating a
dictionary. Many dictionaries today use larger and broader corpora. The SemCor files contain over
20,000 tagged words across 352 files. The appendix contains a short table indicating what kinds of text
sources SemCor tags. 186 of these files contain tagged nouns, verbs, adjectives, and adverbs. The
remaining 166 files only tag verbs. Every tag contains the part of speech, the lemma, and the correct
WordNet sense. This makes SemCor extremely useful for researchers using WordNet. SemCor provides
a professionally tagged resource to compare the accuracies of word sense disambiguation algorithms.
3.5 SemEval
SemEval is a language processing competition that has occurred approximately every three
years since 1998. Each competition requires several “tasks” in which participants can compete. These
tasks range from translating text to word sense disambiguation in many languages. This project focuses
on the “all words” task. The goal of the all words task is to assign the correct sense to every given word
in the supplied text. Investigating the work done in these competitions is an optimal way to compare
the results of the author’s project with other techniques.
Each SemEval competition has a slightly different focus and goal. The first SemEval competition
took place in 1998 and set the stage for the competition (ACL-SIGLEX, 2011). The original goal was to
evaluate which senses and algorithms work best. This original competition used Hector as the reference
dictionary. The second SemEval competition took place in 2001. This competition separated the
competition into tasks, including the all-words task, and introduced several languages as separate tasks.
This is the first SemEval competition using WordNet, which was version 1.7 at the time. The third
SemEval competition took place in 2004. There was the addition of several tasks on top of the tasks
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 25
available in SemEval 2. The reference dictionary was WordNet 1.7.1. The fourth SemEval competition
took place in 2007. The sponsors took a critical look at the previous two competitions and realized that
WordNet is too detailed. Many of the senses for a word are so subtle that even humans have a hard
time choosing the correct sense. This is why the best algorithms in the second and third competitions
had a worse accuracy than during the first competition. Therefore, the sponsors took the WordNet 2.1
definitions and grouped similar senses together. When evaluating any of the tasks, any sense in the
correct group was correct as a “coarse” result. The fifth SemEval competition took place in 2010. This
time the goal is to use the same algorithm across several languages. Of course, this means the reference
dictionary was different for every language. To account for this, a common representation of all the
various WordNet resources was created. This “WordNet Lexical Markup Framework” was provided for
in the trial data for the all words task. The tasks and goals of the SemEval competition have definitely
evolved and changed over 12 years.
3.6 OntoNotes
The SemEval 2010 competition also saw a different English reference dictionary for some of the
tasks. Many of the tasks still use WordNet, but some started to use OntoNotes. OntoNotes is a
collaborative effort between Raytheon BBN Technologies, the University of Colorado, the University of
Pennsylvania, and the University of Southern California's Information Sciences Institute (Raytheon BBN
Technologies, the University of Colorado, the University of Pennsylvania, and the University of Southern
California's Information Sciences Institute , 2011). The idea is to make a resource as available as
WordNet, but more accurate and useful for language processing. The past decade has shown several
weaknesses and lessons from WordNet, so the goal of OntoNotes is to overcome them. OntoNotes
originally started with the WordNet glosses and groups them together like in SemEval 2007. Many of
the semantic relations are still in development as a “PropBank” component and should be a much richer
resource according to the advertisements. Currently, it is possible to translate between WordNet and
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 26
OntoNotes for research purposes (Zhong, Ng, & Chan, 2008) (Patwardhan, Banerjee, & Pedersen,
UMND1: Unsupervised Word Sense Disambiguation Using Contextual Semantic Relatedness, 2007).
Keep in mind that OntoNotes is slowly evolving to a different resource and may be just as useful in the
future. This author’s project does not use OntoNotes, but moving towards OntoNotes may make the
approach in the project much more successful at a later time.
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 27
Chapter 4: Introduction to Semantic Relations
A semantic relation uses some aspect of a word to compare two words. There are several
possible semantic relations since there are several aspects to a word. Many early researchers focus on a
single semantic relation. The all words tasks in the SemEval competitions use the “most common” sense
as their baseline (ACL-SIGLEX, 2011). Wu and Palmer look at hypernyms (Wu & Palmer, 1994). Many
later researchers use more than one semantic relation since each relation has its strengths and
weaknesses. For example, Zhang uses frequency, hypernyms, and domain to disambiguate nouns
(Zhang, Zhou, & Martin, 2008). Many researchers found that they achieved better results because they
used several different relations.
In this project, all the semantic relation information comes from WordNet. The algorithm then
calculates a number using an equation specific to that relation. For example, the equation comparing
frequency would be different than an equation comparing synonyms. They may be similar, but the
details of the equation are slightly different. Also, the equation could be different for each part of
speech. A noun will have different similarities to a second noun than it would to a verb. In this project,
there are six semantic relations: frequency, hyponyms, coordinate sisters, domain, antonyms, and
synonyms.
4.1 Frequency
Frequency is how often a definition appears in context compared to other words. Most
definitions in dictionaries are in order. The most “frequent” or “common” definition is the first sense.
The least frequent definition is the last sense. The question is how to take advantage of this
information. As mentioned before, the baseline for many word sense disambiguation projects, including
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 28
SenseEval, is to compare the performance to an answer using the first sense for all words (ACL-SIGLEX,
2011). Many methods use the most common sense when other relations don’t give a clear answer.
Frequency is a very common and useful comparison.
Conceptually, there are several possible ways to calculate the Frequency for two words. The
developers of WordNet kept track of how many times a sense was used in the corpus used to develop
WordNet. As expected, the sense with the most occurrences is the first definition. The number of
occurrences for each sense is available to the users of WordNet. Some researchers, including this
project, use the number of occurrences of a sense and divide it by the total number of occurrences of
that word (Zhang, Zhou, & Martin, 2008). However, not all words appear in the WordNet corpus. To
account for these exceptions, a linear distribution is applied. The equations are below.
𝑆𝑒𝑛𝑠𝑒𝐶𝑛𝑡(𝑤)
,
𝑇𝑜𝑡𝑎𝑙𝐶𝑛𝑡 (𝑤) > 0
𝑇𝑜𝑡𝑎𝑙𝐶𝑛𝑡(𝑤)
𝐹𝑟𝑒𝑞 (𝑤) =
𝑆𝑒𝑛𝑠𝑒𝑇𝑜𝑡𝑎𝑙 (𝑤) − 𝑆𝑒𝑛𝑠𝑒 (𝑤) − 1
,
𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
𝑆𝑒𝑛𝑠𝑒𝑇𝑜𝑡𝑎𝑙(𝑤)
{
SenseCnt(w): The number of times this was referenced in WordNet corpus
TotalCnt(w): The total number of references for this word in WordNet corpus
SenseTotal(w): The total number of senses for this word
Sense(w): The sense number of the word currently in use
Equation 4: Semantic Relation Equation for Frequency
Notice that frequency is easily applied to a single sense of a word. If the word is “frequency”, as
below, the value for sense 1 is 0.48. However, a semantic relation compares two words, not just one.
To account for this, one adds the calculated frequency of the two words. If the first word is the first
sense of frequency and the second word is the second sense of frequency, then the calculated
frequency is 0.48+0.44=0.92. In this fashion, the genetic algorithm uses frequency in the same manner
as any of the other semantic relations.
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 29
1. (12) frequency, frequence, oftenness -- (the number of occurrences
within a given time period)
a. 12/25 = 0.48
2. (11) frequency, relative frequency -- (the ratio of the number of
observations in a statistical category to the total number of
observations)
a. 11/25 = 0.44
3. (2) frequency, absolute frequency -- (the number of observations in a
given statistical category)
a. 2/25 = 0.08
Example 3: Example using the Frequency Semantic Relation Equation
4.2 Hypernym
A hypernym is a more generic way of saying a word. For example, a more generic way of saying
man would be human. A more generic way of saying human would be organism. Note that some words
have a different hypernym depending on definition. A “bank” next to the river has a different hypernym
than a financial “bank.” Finding the hypernym of every sense creates a tree-like structure of hypernym
relations. Many researchers take advantage of this tree in many different ways. This project uses Wu
and Palmer (Wu & Palmer, 1994).
Wu and Palmer account for several aspects of the hypernym tree. When comparing two words,
the point where the hypernyms of both words match is the most significant ordinate or lowest common
subsumer. The depth to this point indicates how specific the two words are. The distance between the
two words in the tree indicates how similar the two words are. Wu and Palmer accounts for both so
that the similarity of two words is just as important as how specific the two words are.
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 30
entity
𝐻𝑦𝑝(𝑤1 , 𝑤2 ) =
3 hypernyms…
organism
person
animal
male
D
MSO
female
A
boy
B
girl
2×𝐷
A+𝐵+2×𝐷
MSO: most specific common
hypernym
D: length from the root to MSO
A: length from w1 to MSO
B: length from w2 to MSO
2×5
2+2+2×5
𝐻𝑦𝑝(𝑏𝑜𝑦, 𝑔𝑖𝑟𝑙) ≈ 0.714
𝐻𝑦𝑝(𝑏𝑜𝑦, 𝑔𝑖𝑟𝑙) =
Equation 5 and Example 4: Semantic Relation Equation and Example for Hypernyms
In the example above, there are three base words: boy, girl, and animal. The distance between
the words boy and girl is 4. The distance to the most significant ordinate of these two words, person, is
5. This makes the hypernym score between these two words 0.714. Note that comparing animal and
boy is 0.666, which is lower than the comparison of boy and girl. Animal and boy may be just as similar,
but they are more generic. Wu Palmer accounts for both the similarity of two words and how similar
they are.
4.3 Coordinate Sisters
Two words that have the same hypernym are coordinate sisters. Looking at these separately
could provide different and potentially useful insight than when only using Wu and Palmer strategies.
This gives extra emphasis on closely related terms whether they are generic terms or not.
The equation for coordinate sisters is simple. Either the two words are coordinate sisters, or
they are not. Therefore, the equation returns 1 when they are coordinate sisters and 0 when they are
not. In the example tree below, male and female are coordinate sisters. Boy and animal are not
coordinate sisters.
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 31
entity
3 hypernyms…
organism
animal
person
male
female
boy
girl
𝐶𝑜𝑜𝑟(𝑤1 , 𝑤2 ) = {
1, 𝐶𝑜𝑜𝑟(𝑤1 ) ∋ 𝑤2
0, 𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
w1 and w2: the two words being compared
Coor(w*): all of the coordinate sisters of the
word
Coor(w1) ∋ w1: if w2 is any of the coordinate
sisters of w1
Equation 6 and Example 5: Semantic Relation Equation and Example for Coordinate Sisters
4.4 Domain
Domain is the word or collection a word belongs to. For example, several programming and
computer terms belong to the domain computer science. Several terms describing plants and gardening
belong to the domain botany. This can make a big difference when looking at senses. The idea is that
some text describing something in a domain will use several words from a domain in the descriptions. If
the domain of the text is around computer science, then a sense describing a computer disk drive is the
most likely to be the correct sense for disk. Very domain specific text can take advantage of this
information, which is why SemEval 2010 has a domain specific task. Domain can make a big difference
in some situations.
Computer Science
Buffer, drive, cache,
program, software
Sports
Skate, backpack, ball,
foul, snorkel
𝐷𝑜𝑚(𝑤1 , 𝑤2 )
1, 𝐷𝑜𝑚(𝑤1 ) = 𝐷𝑜𝑚(𝑤2 )
= {
0, 𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
w1 and w2: the two words being
compared
Dom(w*): the domain of the word given
Equation 7 and Example 6: Semantic Relation Equation and Example for Domain
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 32
The calculation for domain is very simple. It is either part of the domain, or it is not. There are
no sub-domains or generic domains available in WordNet. Therefore, the equation for domain below
returns a 1 if the words are in the same domain, otherwise it returns a 0. For example, comparing
computer and cache returns 1. Comparing computer and baseball will return 0.
4.5 Synonym
A word is a synonym of second word if one can interchange the words without changing the
meaning of the sentence. A different way of defining synonyms is to say that two words are synonyms if
they have the same meaning. WordNet takes advantage of this fact by grouping all synonyms into a
SynSet (Synonym Set). Any word in the SynSet returns the same exact meaning. Even though two
words have the same meaning, people will interchange synonyms because one synonym “sounds”
better. Sometimes people simply want a little variety in their sentences and pull out their thesaurus.
Whatever the reason is, comparing the synonyms of senses could provide insight into which sense is
correct.
Every word is part of a SynSet, which is a synonym that has the exact same meaning. Just like a
thesaurus, WordNet also keeps track of SynSets that have similar meanings. If two words are Synonyms
of each other, exact or similar, then the equation below returns 1. Otherwise, the equation below
returns 0.
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 33
1. computer, computing machine,
computing device, data
processor, electronic
computer, information
processing system
2. sport, athletics
3. frolic, lark, rollick, skylark,
disport, sport, cavort, gambol,
frisk, romp, run around, lark
about
𝑆𝑦𝑛(𝑤1 , 𝑤2 ) = {
1, 𝑆𝑦𝑛(𝑤1 ) ∋ 𝑤2
0, 𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
w1 and w2: the two words being
compared
Syn(w*): all of the synonyms of the
word
Syn(w1) ∋ w2: if w2 is any of the
synonyms of w1
Equation 8 and Example 7: Semantic Relation Equation and Example for Synonyms
4.6 Antonym
An antonym has the opposite meaning. Anything that is an opposite is an antonym. Some
examples are: black vs. white, angel vs. demon, good vs. evil, vague vs. clear, antonym vs. synonym, etcetera. This would work well for comparison papers. One item did this while the other paper did the
opposite. Many researchers who look at synonyms often look at antonyms because these same
comparison papers also describe any similarities.
As with many of the other semantic relations, two words are either antonyms of each other or
are not. Therefore, the equation below returns the value 1 if two words are antonyms. Otherwise, the
equation returns 0.
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 34
1. good, goodness
a. evil, evilness
2. pure
a. defiled, maculate
3. vague
a. clear, distinct
1, 𝐴𝑛𝑡(𝑤1 ) ∋ 𝑤2
𝐴𝑛𝑡 (𝑤1 , 𝑤2 ) = {
0, 𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
w1 and w2: the two words being
compared
Ant(w*): all of the antonyms of the
word
Ant(w1) ∋ w2: if w2 is any of the
antonyms of w1
Equation 9 and Example 8: Semantic Relation Equation and Example for Antonyms
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 35
Chapter 5: A Word Sense Disambiguation Genetic Algorithm
Word Sense Disambiguation for all words in a file creates a large solution space. For example,
suppose that there exists a file with five paragraphs of information. Each of these paragraphs could
have 100 words, which means the file has 500 words. Each word can have many definitions, so assume
there are five definitions per word. This makes 5100 = 7.888 * 1069 possible word sense combinations for
a paragraph or 5500 = 3.055*10349 word sense combinations for the entire file. This solution space is far
too large to check every word sense combination. If there is a way to measure the accuracy of a word
sense combination, genetic algorithms provide a way to check a subset of these solutions in order to
obtain a good solution. This solution will not be “the solution,” but the solution should be close.
The genetic algorithm is based on Darwin’s theory of evolution to solve optimization problems.
The general idea is to start with a set of solutions, using that set of solutions to make better solutions,
and only keeping the best solutions. In genetic algorithms, chromosomes define all of the information
necessary to define a solution. The individual pieces of the chromosome are genes that define a certain
aspect of the solution. In this case, the chromosome defines the word senses for the file. Each gene
represents the chosen word sense for a single word. The generic process of creating “better”
chromosomes, or solutions, with genetic algorithms is below.
1. Start with a set of solutions (1st Generation)
2. Take original “parent” solutions and combine them with each other to create a new set of
“child” solutions (Mating)
3. Introduce some random changes in case solutions are “stuck” or are all the same (Mutation)
4. Somehow measure the solutions (Cost Function) to evaluate the best solution
5. Repeat starting with Step 2
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 36
Genetic Algorithm implementations can vary in many respects. The preceding steps describe
the most common case at a very high level. Most genetic algorithms have three main parts. They are:
mating, mutation, and cost function. The preceding steps above do not describe how to implement
mating, mutation, and cost function. The implementation of these portions has different strengths and
weaknesses depending upon the problem. It also does not describe how to select chromosomes for
each step. One should expect the implementation to change for every problem, and most authors will
have a cost function, a mating phase, and a mutation phase as part of their genetic algorithm.
5.1 Evolution of the Cost Function
The cost function is possibly the most important part of a genetic algorithm. It is the portion of
the algorithm that determines which solution is the “correct” or “better” solution. How to determine
the best answer depends upon the problem and on the approach. If the cost function’s approach
incorrectly compares solutions, then the genetic algorithm will lead to an increasingly incorrect answer.
For example, if the problem is to find the highest point on a mountain, the cost function needs to
correctly determine which point is higher in elevation. Assume that the cost function compares two
areas according to the slope of the hillside under the assumption that the slope on the top of the
mountain is zero. In this case, the genetic algorithm will not be able to correctly compare two
mountains or plateaus on the map. In addition, it does not matter how well the rest of the genetic
algorithm functions because the results will most likely be incorrect. If the genetic algorithm is to find a
“good” answer, then the cost function must be able to determine which answers are correct in most
cases.
For this project, the cost function must be able to compare the definitions of a word and
determine which definition is most likely to be correct. All of the preceding semantic relation equations
are different measurements that can be part of the cost function. Each of these measurements has its
strengths and weaknesses. A large part of this author’s project involves examining these measurements
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 37
and determining how to use them most accurately. The sections below begin by examining each
semantic relation. The author then incorporates them into a cost function and analyzes the results. This
leads to several variations of the cost function using the knowledge and results of the last cost function.
The last cost function is the one the genetic algorithm uses.
5.1.1 Notes on Sampling
Earlier in this report, the author explored SemCor and WordNet. The SemCor files list the
correct WordNet senses for the given text. Since this project needs to compare incorrect senses with
correct senses, there is a need for many other possible sense combinations. Throughout this project, a
solution is a specific selection of sense combinations. The number of senses that match the senses in
the SemCor file indicate how “good” the solution is. With that in mind, the SemCor selection is the best
solution, and the worst solution matches none of the SemCor senses. The author has noted a variety of
solutions and several levels of correctness.
The problem surfaces when examining the number of possibilities. Each SemCor file contains
approximately 2000 words. Of these, 1100 to 1300 words have a WordNet definition. These definitions
may be a noun, verb, adjective, or adverb. Assuming that the part of speech is already identified for
each word, each of these defined words has the following average number of definitions: noun 1.24,
verb 2.17, adjective 1.4, and adverb 1.25. If every defined word were a noun, the number would be
1.241200 = 1.2765*10112 possible combinations per SemCor file. If there were an equal distribution for
each part of speech, this would be 1.5151200 = 3.1272*10216 possible combinations per SemCor file. This
is an inordinate number of combinations. To further clarify this, imagine that if a solution were
calculated every nanosecond; it would require 9.9164*10199 years to investigate each combination. It
follows that a subset of solutions must represent all of the solutions. The selected subset areas are:
1. Near the correct solution
2. Randomly picked solutions
3. Near the most frequent senses
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 38
4. Random solutions weighted towards the most frequent senses
5. Min/Max areas for each semantic relation via a Mutation method
The author has selected the five primary areas to represent all of the possible solutions. Each
area is a starting point. Every solution begins in one area and randomly changes a percentage of the
word senses toward another area. The first area is near the correct solution. These answers give insight
regarding how to reach the correct solution. The second area contains randomly selected senses to
provide variety. The third area is located near the most frequent senses since the most frequent sense
is a common baseline. The fourth area contains “weighted” randomly selected solutions. This provides
answers between the randomly selected area and the most frequent area. The weighted random
equation and an example are formulated below. The last area describes the boundaries of each
individual semantic relation. A function that eventually became one of the mutation functions defines
these boundaries by building a solution to match the given value for a semantic relation. (See the
mutation section for further explanation.) Using 3000 solutions and using these five areas gives variety
and focus to address most of the cost function behavior.
𝑊𝑒𝑖𝑔ℎ𝑡𝑒𝑑𝑃𝑟𝑜𝑏 (𝑤, 𝑠) =
𝑇𝑜𝑡𝑎𝑙𝑆𝑒𝑛𝑠𝑒 (𝑤) − 𝑠 + 1
∑𝑇𝑜𝑡𝑎𝑙𝑆𝑒𝑛𝑠𝑒(𝑤) 𝑠
w and s: the word and sense
TotalSense(w): the number of senses for this word
Equation 10: The Weighted Probability Equation
If the word has 4 senses, the probability of returning the 1st sense is:
4−1+1
1+2+3+4
𝑊𝑒𝑖𝑔ℎ𝑡𝑒𝑑𝑃𝑟𝑜𝑏 (𝑤, 1) = 0.4
𝑊𝑒𝑖𝑔ℎ𝑡𝑒𝑑𝑃𝑟𝑜𝑏 (𝑤, 1) =
Example 9: An example using the Weighted Probability Equation
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 39
5.1.2 Semantic Relation Investigation
Studying semantic relations behavior is the first step to making the cost function. Each semantic
relation is a small part of the cost function. The behavior of the semantic relations in this section has a
major influence on the cost function variations in the following sections. Therefore, the coverage and
representation for each semantic relation is crucial to the process. The variety and boundaries of the
semantic relation indicate where and how each semantic relation influences the solution. This includes
the parts of speech which the semantic relation represents. To illustrate, hypernyms are available solely
for nouns and verbs in WordNet, and therefore the relation is useless when comparing adjectives and
adverbs. In addition, the hypernym trees for nouns do not overlap with hypernym trees for verbs. For
this reason the results for every part of speech combination is unique within each semantic relation.
The cost function can then account for each semantic relation and part of speech combination
separately.
A second thing to note is that each semantic relation compares two words in order to return a
value. There are several possible word pair combinations in the given text. The challenge is to
determine which combinations are important. Various researchers try different techniques. Zhang finds
every word combination in a paragraph (Zhang, Zhou, & Martin, 2008). The idea is that a single
sentence does not contain enough information to properly determine the correct word sense with
semantic relations. A paragraph would have more information because it has several related sentences.
Zhang’s solution contains one drawback. Very large paragraphs have an extremely large number of
combination pairs. In addition, the SemEval competitions do not indicate where the paragraph
boundaries are located. The competitors in many recent SemEval competitions use a sliding window
technique for this reason (Patwardhan, Banerjee, & Pedersen, UMND1: Unsupervised Word Sense
Disambiguation Using Contextual Semantic Relatedness, 2007) (Bosch, Hoste, Daelemans, & Den, 2004)
(Mihalcea & Csomai, 2005). The first step is to remove all the words that don’t have definitions in
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 40
WordNet. Then use all the combinations “n” words before and after the target word. The number of
words before and after varies from three words to five words in many papers. This project checks for
words ten words before and after the target word. This is a compromise between a paragraph and the
sliding window technique. Each word has a window with 45 possible combinations while a paragraph
can easily have thousands of possible combinations. Note that each combination uses the semantic
relation equations below to return a value. The overall semantic relation behavior is the average of all
the individual values.
One of the most transparent ways to view behavior is by means of a graph. With graphs of the
semantic relation, it is easier to see the estimated solution space of the relation, the boundaries of the
relation results, the variety of solutions that give a specific result from a semantic relation equation, and
the coverage. Therefore, a scatter plot with 3000 different solutions is available for every semantic
relation. These graphs have the semantic relation as the independent variable. The accuracy of a
solution is the independent variable. In several graphs, each of the individual sample areas is visible.
The graph below uses frequency and indicates the location of five sample areas. The sections below do
not show the individual areas. However, all these areas are very important for finding the boundaries of
each semantic relation.
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 41
1) Correct Area
3) Most
Frequent Area
Average
Solution Value
2) Random Area
4) Weighted
Random Area
5) Max
Mutation Area
5) Min
Mutation Area
Figure 1: An example graph indicating the various sample areas
5.1.2.1 Frequency Behavior
The graphs for two part of speech combinations for frequency are included below. All part of
speech combinations have approximately the same shape. The lowest frequencies have very low
accuracy, but more than zero. This implies that some words only have one definition or use the least
frequent definition. The highest frequencies have ~75% accuracy. This solution would be the typical
“baseline” solution recognized by many authors. The area between them contains a distinct point
where a small range of frequencies contain “good” solutions. However, this small range of frequencies
also contains the widest range of accuracies. This means that it will take more than frequency to sort
out the subset of solutions in this range. The five sample areas are clear and appear to cover several
boundaries of the semantic relation. All in all, frequency looks like a promising semantic relation as an
individual measurement.
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 42
Distinct
Maximum
Large Range at
Correct Solution
Figure 2: The graph of the frequency semantic relation using a noun-noun part of speech combination
Distinct
Maximum
Large Range at
Correct Solution
Figure 3: The graph of the frequency semantic relation using an adjective-adjective part of speech
combination
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 43
5.1.2.2 Hypernym Behavior
There are only two part of speech combinations that work for hypernyms: comparing two nouns
or comparing two verbs. In both cases the sample areas are not as distinct. The noun-noun
combinations appear to have more solutions on the left side of the correct solution. This may explain
why many researchers who use hypernyms can achieve greater accuracy by maximizing hypernyms
scores. The bulk of the solutions have a lower hypernym score and other researchers don’t access these
exceptions very often. In either case, the noun-noun hypernym graph has a distinct point around the
correct solution near the middle of the range. The verb-verb hypernym graph does not display the same
distinct point that the noun-noun hypernym displays. It appears to be inversely proportional to
accuracy. This implies that actions are not as related as the subjects. Both graphs indicate that there is
a relation between hypernyms and accuracy. Both graphs also indicate that the score around the
correct solution has the widest range of accuracies. Of the two graphs, the noun-noun version looks
more useful.
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 44
Distinct
Maximum
Very Large and
Dense Range at
Correct Solution
Figure 4: The graph of the hypernym semantic relation using a noun-noun part of speech combination
Distinct
Maximum
Very Large and Very
Dense Range at
Correct Solution
Figure 5: The graph of the hypernym semantic relation using a verb-verb part of speech combination
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 45
5.1.2.3 Coordinate Sister Behavior
Both coordinate sisters and hypernyms rely on the same hypernym tree, so both of them also
have the same part of speech limitations. Coordinate sisters can only compare two nouns or two verbs.
In this case, the graphs are somewhat similar. The noun-noun combination has a distinct point around
the correct solution in the middle of the coordinate sister range. The verb-verb version has the distinct
point near the beginning of the range. The verb-verb graph indicates the correct solution is not as close
to zero as the hypernym graphs. Both graphs show that the area around the solution has a majority of
the solutions and the widest range of accuracies. Overall, the coordinate sister results are slightly
different than the hypernym results, which may prove useful later on.
Distinct
Maximum
Very Large and Very
Dense Range at
Correct Solution
Figure 6: The graph of the coordinate sister semantic relation using a noun-noun part of speech
combination
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 46
Distinct
Maximum
Very Large Range at
Correct Solution
Figure 7: The graph of the coordinate sister semantic relation using a verb-verb part of speech
combination
5.1.2.4 Domain Behavior
Few words have a domain in WordNet. This fact is evident in the graphs for domain because the
possible values show up as columns. Perhaps a different SemCor file that is more domain specific may
have a larger variety of results, but br-g23 only has a few. A majority of the given solutions have an
average domain value below 0.001. In addition, the correct solution is not unique among the given
solutions. This makes it very hard to see a correlation between domain and accuracy if one indeed
exists. Overall, domain does not look promising at this point. At least it does not look promising for this
SemCor example.
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 47
Not Very Distinct
Maximum
Very Large Range at
Correct Solution
Figure 8: The graph of the domain semantic relation using a noun-noun part of speech combination
Not Very Distinct
Maximum
Very Large Range at
Correct Solution
Figure 9: The graph of the domain semantic relation using an adverb-adverb part of speech combination
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 48
5.1.2.5 Synonym Behavior
Synonyms show similar issues that domain did, except on a lesser scale. There simply are not
enough synonyms in the br-g23 SemCor file to provide a variety of results. In the verb-verb example
below, a large majority of the answers have approximately the same two values. The larger group
happens to contain the correct answer. There is a distinct maximum point, but this is a 50% chance in
the first place. As for the adjective-adjective example, it does have more than two solutions. However,
it shows very little correlation between synonyms and accuracy. It is unlikely that synonyms will be
useful in the cost function later on.
Distinct
Maximum
Very Small
Variation
Figure 10: The graph of the synonym semantic relation using a verb-verb part of speech combination
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 49
Not Very Distinct
Maximum
Very Small
Correlation
Figure 11: The graph of the synonym semantic relation using an adjective-adjective part of speech
combination
5.1.2.6 Antonym Behavior
If synonyms do not look very promising, then antonyms will probably have the same results.
The two are extremely similar in nature, so each of them should have similar strengths and weaknesses.
Antonyms do not have sufficient variety in the results to be reliable. None of the solutions have an
antonym average above 0.005! There is no distinct point near the correct solution. The correlation
between antonyms and accuracy is very hard to determine. Overall, antonyms will perform as badly as
synonyms and will probably not be useful in the cost function.
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 50
Not Very Distinct
Maximum
Very Small
Correlation
Figure 12: The graph of the antonym semantic relation using a noun-noun part of speech combination
Very Small
Correlation
Very Small
Variation
Figure 13: The graph of the antonym semantic relation using an adjective-adjective part of speech
combination
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 51
5.1.3 The Optimal Cost Function
At this point there are several semantic relation equations and graphs. The author’s plan is to
use those equations in such a way so that the best solutions have the highest costs while the lower
accuracy solutions have lower costs. The goal of the genetic algorithm is to aim for the highest cost
solution in every generation. Ideally, the highest cost possible will be the correct solution and nothing
but the correct solution. A different way of describing it is, “The accuracy of the solutions is
proportional to the cost of the solution with little or no variance near the maximum solution.” The
scatter plot of the cost functions should resemble the one below.
Max Cost is
Correct Solution
Very Little Variation
in Accuracy near
Max Cost
Figure 14: The graph indicating the shape of the ideal cost function
5.1.4 Cost Function Method 1: Simple Addition
The easiest way to create a cost function is to add the average values of each semantic relation.
This provides a good starting point. It is not, and was never intended to be, the final cost function. It
serves to determine how much the cost function needs to change in order to make an optimal cost
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 52
function. This provides a baseline to determine whether the future solutions actually improve beyond
the simplest solution. The resulting graph is below.
Correct Answer Not
the Maximum Cost
Maximum Cost,
≈75% Accuracy
Figure 15: The graph indicating the shape of Cost Function Method 1
The result is not ideal. The highest cost solution is not the correct solution, but variance at this
highest point is small. The result takes the same shape as the frequency semantic relation. This is not
surprising considering the fact that frequency has the highest average values by a significant amount.
This also means that the answer this cost function converges to is the most frequent sense. Considering
this is the most common baseline, this answer is not an acceptable one.
5.1.5 Cost Function Method 2: Regression
Thorough examination of the semantic relations will reveal three main points. The maximum
score in all the semantic relations is not the correct answer. For example, the maximum frequency
score gives an accuracy of ~75%. If each semantic relation had the maximum point as the correct
solution, then the addition of the semantic relations should be the correct solution. Some semantic
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 53
relations appear to relate to accuracy better than others. Antonyms, for example, do not appear to
have an effect on accuracy. Those semantic relations which do not relate as well have a higher chance
of influencing the cost function to an incorrect solution. The semantic relations vary a little between
different SemCor files. A good cost function would need to account for all of these points in order to
succeed.
One possibility is polynomial regression. Take the semantic relation values and accuracies and
fit a 6th order polynomial to the result. Then use that equation to transform the average semantic
relation value. This would make the maximum value at or close to the correct answer. To account for
the multiple SemCor files, simply make a regression of the regressions. Then this average regression
would account for as many of the SemCor files as possible. Multiply by the R2 value of this average
regression to adjust for semantic relation variance. A step by step example is below.
1. Take several samples for one SemCor file
2. Find the regression for one semantic relation.
Figure 16: A graph with an example regression equation
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 54
3. Find the regression of regressions of that semantic relation from multiple SemCor files.
Figure 17: A graph indicating an example of the regression of multiple regressions for frequency
4. Use the R2 value as a multiplier for the semantic relation
5. Calculate the total weighted cost with the following equation. It adds the transformed semantic
relation value for every word combination.
𝑊𝑒𝑖𝑔ℎ𝑡𝑒𝑑𝐶𝑜𝑠𝑡(𝑠)
=
∑ 𝐴𝑣𝑔 ( ∑
𝑆𝑒𝑚𝑅𝑒𝑙
∑ 𝑆𝑒𝑚𝑅𝑒𝑙𝐸𝑞𝑛(𝑆𝑒𝑚𝑅𝑒𝑙, 𝑊𝑜𝑟𝑑1 , 𝑊𝑜𝑟𝑑2 ))
𝑊𝑜𝑟𝑑1 𝑊𝑜𝑟𝑑2
S: The current solution
Word1 and Word2: Every word in the solution
SemRel: Every semantic relation
SemRelEqn: Use the words in the correct semantic relation equation
Avg: Find the average value then transform with the regression equation
Equation 11: The equation for Cost Function Method 2
An examination of the resulting cost function is better, according to the following graph. The
correct solution has a higher cost than the first method, but it still is not the highest. There appear to be
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 55
several answers that have a higher cost. Also, there is a large variance near the maximum with as low as
65% accuracy. This method remains insufficient.
Correct Answer
Near Maximum Cost
Maximum Cost,
Several Accuracies
Figure 18: The graph indicating the shape of Cost Function Method 2
5.1.6 Cost Function Method 3: Proportional Placement in a Range
The regression for some semantic relations, such as frequency, work fairly well. However,
regression has its weaknesses. Any extraneous points tend to have a large influence on the regression.
The maximum value of the curve does not match the correct answer. In most cases it only varies a
slightly as in Figure 18.
The results between the SemCor files make the difference here. Hypernyms, for example, do
not always have the same range of values for every SemCor file as shown in the graph below. The graph
illustrates some of the files that have the best R2 values for the hypernym noun-noun regression. The
correct hypernym value for one SemCor file can be the worst answer for another file. In this case, 0.26
is the best answer for br-g16, and it is the worst answer for br-e30. This skews the cost function despite
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 56
the fact that the original regressions fit better than most of the semantic relations. As a result of this
skewing, one might determine that something other than regression works better for hypernyms.
Correct Solution Varies
Same Cost, Opposite
Effect Between Files
Figure 19: A graph showing an example of the weaknesses of multiple hypernym regressions
The graph above illustrates that regression between files fails. It also implies a trend about
hypernyms. The ranges change, but the correct answer is always at or near the middle of that range.
For frequency, the correct answer is closer to the maximum value of the range. In the preceding
samples, one of the areas is the boundaries of a semantic relation. The function that locates these
boundaries is already available, so it is possible for the genetic algorithm to find the range. Since
dynamically finding the range is not an issue, an alternate possibility is to determine what proportion in
that range is typically correct. Then an equation based on these proportions will provide a better
transformation than would regression. The steps and example below illustrate how to find these
equations.
1. Take several samples for one SemCor file
2. Record the min, max, and ratio of the correct answer for each Semantic Relation
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 57
3. Find the average and standard deviation of the correct solution ratio for each semantic relation
and part of speech combination from multiple SemCor files
4. Use four lines to transform the results. The lines connect five crucial points: the min, the max,
the average of the correct solution, and the two points twice the standard deviation away from
the average. Note that the genetic algorithm will find the range dynamically. A graphical
example is below.
(Avg, stdDev2)
(Avg – 2*stdDev, 0.9*stdDev2)
(Avg + 2*stdDev,
0.9*stdDev2)
(Max, 0)
(Min, 0)
Figure 20: A graph showing an example using Cost Function Method 3
5. Calculate the total cost with the following equation. It adds the transformed semantic relation
value for every word combination.
𝑊𝑒𝑖𝑔ℎ𝑡𝑒𝑑𝐶𝑜𝑠𝑡 (𝑠)
=
∑ 𝐴𝑣𝑔 ( ∑
𝑆𝑒𝑚𝑅𝑒𝑙
∑ 𝑆𝑒𝑚𝑅𝑒𝑙𝐸𝑞𝑛(𝑆𝑒𝑚𝑅𝑒𝑙, 𝑊𝑜𝑟𝑑1 , 𝑊𝑜𝑟𝑑2 ))
𝑊𝑜𝑟𝑑1 𝑊𝑜𝑟𝑑2
S: The current solution
Word1 and Word2: Every word in the solution
SemRel: Every semantic relation
SemRelEqn: Use the words in the correct semantic relation equation
Avg: Find the average value then transform with the proportional equation
Equation 12: The equation for Cost Function Method 3
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 58
Both regression and this technique find the most likely values that contain the correct solution.
The difference is that this technique takes into account the changing range between the SemCor files.
As such, this method has fewer solutions that have a higher score than the correct solution, as shown
below. Just like regression, however, there is a large variance in accuracies around the highest cost.
More exploration is necessary to eliminate these false positives.
Correct Answer Near
the Maximum Cost
Maximum Cost,
Several Accuracies
Figure 21: The graph indicating the shape of Cost Function Method 3
5.1.7 Cost Function Method 4: Add Sense Distribution
The method above is very promising. Its main weakness is that the highest cost can be a variety
of answers, ranging anywhere from a very inaccurate answer to the correct answer. All of these
answers provide average semantic relation scores that are in the optimum ranges. However, the lower
accuracy answers have a different sense distribution than the correct answer. They contain a different
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 59
number of senses that use the first sense and a different number of senses that use the second sense.
The steps below describe how to find and apply these distributions.
1. Take several SemCor files and find the average percentage of each sense
2. Find the error using the following equation. It adds the error for each sense.
𝐸𝑟𝑟𝑜𝑟 (𝑠) = 1 − ∑
𝑆𝑒𝑛𝑠𝑒
𝐴𝑏𝑠(𝑇𝑜𝑡𝑎𝑙𝐶𝑛𝑡 (𝑠𝑒𝑛𝑠𝑒) − 𝑆𝑒𝑛𝑠𝑒𝐶𝑛𝑡(𝑠𝑒𝑛𝑠𝑒))
𝑇𝑜𝑡𝑎𝑙𝐶𝑛𝑡(𝑠𝑒𝑛𝑠𝑒)
S: The current solution
Sense: The sense number
SenseCnt: Find the current number of the given sense
TotalCnt: The expected number of the given sense
Abs: Absolute value
Equation 13: The equation for sense distribution error
3. Multiply the proportional range weighted cost function with this error value
𝑊𝑒𝑖𝑔ℎ𝑡𝑒𝑑𝐶𝑜𝑠𝑡(𝑠) = 𝐸𝑟𝑟𝑜𝑟(𝑆𝑒𝑛𝑠𝑒)
∗ ∑ 𝐴𝑣𝑔 ( ∑
𝑆𝑒𝑚𝑅𝑒𝑙
∑ 𝑆𝑒𝑚𝑅𝑒𝑙𝐸𝑞𝑛 (𝑆𝑒𝑚𝑅𝑒𝑙, 𝑊𝑜𝑟𝑑1 , 𝑊𝑜𝑟𝑑2 ))
𝑊𝑜𝑟𝑑1 𝑊𝑜𝑟𝑑2
S: The current solution
Word1 and Word2: Every word in the solution
SemRel: Every semantic relation
SemRelEqn: Use the words in the correct semantic relation equation
Avg: Find the average value then transform with the proportional
equation
Equation 14: The equation for Cost Function Method 4
The premise behind this additional measurement is to force high cost solutions to have optimal
semantic relation scores and the proper sense distribution. As a result, many of the answers that did
not have the proper sense distribution have lower scores. This separates a large part of the variation at
the maximum cost function score, as shown below. The correct solution is also close to the maximum
score. Overall, this is much closer to the optimal cost function shape than many of the previous
techniques. This looks promising when one assumes that the subset of examples matches the real
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 60
solution space. At this point in the project, the author began to investigate the rest of the genetic
algorithm.
Correct Answer Near
the Maximum Cost
Maximum Cost,
Less Accuracy
Figure 22: The graph indicating the shape of Cost Function Method 4
5.1.8 Cost Function Method 5: Add Semantic Relation Distribution
Despite what the sample solutions show, the genetic algorithm finds solutions that have better
scores than the correct solution. However, these solutions do not exhibit the same accuracy as the
correct solution. These are select solutions that are not part of the sample space and are “optimal” as
far as the cost function is concerned. The last modification accounts for the sense distribution, which
helps remove unwanted solutions such as these. Applying the same concept on the semantic relation
level may remediate these new solutions. The steps for applying this concept are below.
1. Take several SemCor files and find the average semantic relation value for each sense
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 61
2. Find the error using the following equation. It adds the error for each sense.
𝐸𝑟𝑟𝑜𝑟(𝑆𝑒𝑚𝑅𝑒𝑙 )
= 1− ∑
𝑆𝑒𝑛𝑠𝑒
𝐴𝑏𝑠(𝑆𝑒𝑚𝑂𝑝𝑡 (𝑠𝑒𝑛𝑠𝑒) − 𝑆𝑒𝑚𝑉𝑎𝑙(𝑠𝑒𝑛𝑠𝑒))
𝑆𝑒𝑚𝑂𝑝𝑡(𝑠𝑒𝑛𝑠𝑒)
SemRel: The current semantic relation
Sense: The sense number
SemVal: The average semantic relation value for this sense
SemOpt: The optimal semantic relation value for this sense
Abs: Absolute value
Equation 15: The equation for semantic relation distribution error
3. Multiply the semantic relation by this error when calculating the cost function
𝑊𝑒𝑖𝑔ℎ𝑡𝑒𝑑𝐶𝑜𝑠𝑡 (𝑠) = 𝐸𝑟𝑟𝑜𝑟(𝑆𝑒𝑛𝑠𝑒) ∗ ∑ (𝐸𝑟𝑟𝑜𝑟(𝑆𝑒𝑚𝑅𝑒𝑙 )
𝑆𝑒𝑚𝑅𝑒𝑙
∗ 𝐴𝑣𝑔 ( ∑
∑ 𝑆𝑒𝑚𝑅𝑒𝑙𝐸𝑞𝑛(𝑆𝑒𝑚𝑅𝑒𝑙, 𝑊𝑜𝑟𝑑1 , 𝑊𝑜𝑟𝑑2 )))
𝑊𝑜𝑟𝑑1 𝑊𝑜𝑟𝑑2
S: The current solution
Word1 and Word2: Every word in the solution
SemRel: Every semantic relation
SemRelEqn: Use the words in the correct semantic relation equation
Avg: Find the average value then transform with the proportional
equation
Equation 16: The equation for Cost Function Method 5
The concept of applying the distribution on the semantic relation level helps, but not
significantly. Some experiments indicate that this improves the accuracy by only 1% to 2%, a small
percentage compared with the error. This implies that the semantic relation distribution does not
matter as much or that the rest of the cost function already accounts for most of these solutions. At the
very least, this last modification lacks what is necessary to account for the incorrect solutions which the
genetic algorithm finds.
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 62
5.2 Mating
The concept underlying mating is to combine two “parent” solutions somehow and create a
“child” solution. This “child” solution should be similar to the solutions before and can potentially be
better than the parents. Any weaker solutions will eventually “die” and will not mate in the next
solution, as in Darwin’s theory of survival of the fittest. In genetic algorithms terms, this is “elitism.”
Over several generations, the good portions of all the ancestors should collect together into one very
high scoring child.
There are hundreds of ways two parents can mate. In this project, the author has chosen a
dominant gene approach that has worked well in the past (Hausman, A Dominant Gene Genetic
Algorithm for a Transposition Cipher in Cryptography, 2009) (Hausman, A Dominant Gene Genetic
Algorithm for a Substitution Cipher in Cryptography, 2009). The concept is to take the cost function and
apply it on the gene level, or to the individual portions of the solution. This provides a way to determine
which genes are strongest or more “dominant.” These strong genes should have a better chance of
matching the correct solution. Then any children will inherit the dominant genes from both parents.
This makes the child stronger than either parent since the child inherits only the optimal parts of the
solution. Over several generations, this mating technique approaches a strong solution faster than
many other approaches.
In this project, a solution is the current set of senses for each word. Each gene represents the
sense for a single word. To apply the cost function at the gene level, one must keep track of the
semantic relation scores for each word. This means that each word has a contribution to the total cost
function. The words that contribute the most are dominant genes. Since the maximum cost should
represent the best solution, those dominant genes are most likely correct.
There are two ways to mate solutions in this project. When two parents mate, they randomly
choose one of the two methods. Both methods focus on dominant genes. The first method begins with
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 63
the best genes and then combines the mid-range genes. The second method starts with the mid-range
genes and then moves on to the best genes.
5.2.1 Mating Method 1: Mate Top Third
As the title suggests, this mating method sorts the genes by their cost, divides this group into
three, and starts with the highest scoring genes. The top third originate from the first parent. Following
this, the child inherits the top two thirds of the second parent. Any remaining genes are derived from
the first parent. The theory is that the child will inherit the best genes while any other genes are
secondary. This formula focuses on the best possible gene distribution. The detailed steps and example
are below.
1. Find Dominant Genes
Parent 1 Words:
a1
a2
Gene Cost:
11
12
Select the upper 1/3
Dominant Words: a4, a5, and so forth
a3
3
a4
24
a5
15
a6
4
…
…
Parent 2 Words:
b1
b2
b3
Gene Cost:
25
15
5
Select the upper 2/3
Dom. Genes: b1, b2, b4, b6, and so forth
b4
10
b5
8
b6
14
…
…
2. Place Dominant Genes Based on First Parent
Child: *
*
*
a4
a5
*
…
3. Fill in Blanks from Second Parent
Child: b1
b2
*
a4
b6
…
b6
…
a5
4. Fill in any Remaining Blanks from First Parent
Child: b1
b2
a3
a4
a5
5.2.2 Mating Method 2: Mate Middle Third
This mating method also divides the genes into three. This time, however, the first parent
focuses on the middle third. Then the child inherits the top two thirds of the second parent and fills any
remaining genes from the first parent. Sometimes the top genes are very dominant as far as cost is
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 64
concerned, but they are incorrect. Since the genes are so strong, they influence the rest of the solution
incorrectly. By focusing on the middle genes, there is a chance that the middle genes may match those
from the second parent and grow stronger. Over time, any high scoring, incorrect genes will grow
weaker and hopefully change to the correct sense. The detailed steps and example are below.
1. Find Dominant Genes
Parent 1 Words:
a1
a2
Gene Cost:
11
12
Select the middle 1/3
Dominant Words: a1, a2, and so forth
a3
3
a4
24
a5
15
a6
4
…
…
Parent 2 Words:
b1
b2
b3
Gene Cost:
25
15
5
Select the upper 2/3
Dom. Genes: b1, b2, b4, b6, and so forth
b4
10
b5
8
b6
14
…
…
2. Place Dominant Genes Based on First Parent
Child: a1
a2
*
*
*
*
…
3. Fill in Blanks from Second Parent
Child: a1
a2
*
b4
b6
…
b6
…
*
4. Fill in any Remaining Blanks from First Parent
Child: a1
a2
a3
b4
a5
5.3 Mutation
The concept underlying mutations is to give a genetic algorithm a second chance. In many
solutions, there are local maximum solutions that mating can find and get stuck at the local maximum.
For example, if the challenge is to find the highest point on a map, each hill and mountain is a local
maximum solution. The mating function can “get stuck” at the second highest mountain. Had the
mating function turned the other direction at a valley, it would have found the higher mountain. The
mutation function randomly changes some of the solutions so mating may find the other mountain over
time.
In the dominant gene approach, mutations have two objectives. The first objective is to provide
alternate avenues and second chances by providing alternate dominant genes. The second objective is
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 65
to focus on ways to improve the lower cost or recessive genes. The mating function examines only a
part of the solution and does not improve genes that are not already strong. Eventually the mating
function will use all the available “strong” genes and not be able to move toward a stronger solution.
The mutations can modify a recessive gene to make it stronger. This new gene may be strong enough to
become a new dominant gene, a factor which leads to better solution overall. The dominant gene
approach needs to focus on the dominant and recessive genes in order to be successful.
There are four main mutation functions in the sections below. When a solution mutates, it
randomly chooses one of the four mutation functions. In some implementations, the mutated solutions
replace the original solution. In this case, the mutations return a mutated clone to prevent original
solutions from becoming worse. This makes it harder to find alternate avenues, but the solution never
becomes weaker.
5.3.1 Mutation Function 1: Random Mutation
One of the most popular mutations in many genetic algorithms is a random mutation. It is
quick, easy, and finds solutions that are not possible through other means. In this case, it is the main
way to find alternate avenues and paths not naturally found by improving recessive genes. Most of the
time the results are a weaker gene, but this mutation helps in the long run. The steps for using this
mutation are below.
1. Randomly pick a percentage between 0% and 20%
2. Randomly pick that percentage of words from the solution
3. For each of those words, randomly pick one of the available senses
5.3.2 Mutation Function 2: Semantic Relation Score Mutation
The semantic relation score mutation is perhaps the most useful mutation of all the mutations.
This started out as a function to find the boundaries of a semantic relation in the samples section. A
slight modification allows this mutation to modify a solution so it has or is near the given semantic
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 66
relation score. This makes this very useful for moving a solution towards the optimal semantic relation
values, thus creating stronger solutions. It also provides a way for the genetic algorithm to establish the
range of values necessary for the cost function. The steps to the semantic relation score mutation are
below.
1. Start with the given semantic relation average
2. Find the percentage this average is off from the optimal semantic relation score using the
equation below.
𝐶𝑜𝑚𝑝𝑎𝑟𝑖𝑠𝑜𝑛(𝑠) =
𝐴𝑣𝑔(𝑠) − 𝑂𝑝𝑡𝑖𝑚𝑎𝑙(𝑠)
𝑂𝑝𝑡𝑖𝑚𝑎𝑙(𝑠)
S: The current semantic relation
Avg: the average value of the semantic relation for this solution
Optimal: the optimal value for this semantic relation
Equation 17: The equation for comparing the current solution to the optimal solution
3. Randomly pick the comparison percentage of words
4. If the comparison was negative, the semantic relation cost needs to be increased
a. If the semantic relation is frequency, then randomly pick a sense lower than the current
sense. Otherwise use step b.
b. Look at each sense starting at the first sense. Stop and select that sense when the sense
increases the semantic relation cost.
5. If the comparison was positive, the semantic relation cost needs to be decreased
a. If the semantic relation is frequency, then randomly pick a sense higher than the current
sense. Otherwise use step b.
b. Look at each sense starting at the first sense. Stop and select that sense when the sense
decreases the semantic relation cost.
5.3.3 Mutation Function 3: Sense Distribution
The cost function has the following main parts: a semantic relation portion, a sense distribution
portion, and a semantic relation distribution portion. The mutation above covers the semantic relation
portion, so it follows that the second mutation focuses upon the sense distribution. On a higher level,
this mutation finds the current distribution and shuffles the lowest cost genes around to match the
optimal sense distribution. This process achieves two goals. It gives some recessive genes a chance to
become dominant genes and makes the correct distribution to maximize the cost function. This makes
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 67
the sense distribution mutation very useful if it were run several times during the genetic algorithm
process. The steps to the sense distribution mutation are below.
1. Find the number of words extra or missing for each sense compared to the optimal sense
distributions
2. Find the genes that have the lowest gene cost
3. Look at each sense distribution
a. If the current sense distribution has extra words, move the lowest cost genes to a sense
distribution needing words
5.3.4 Mutation Function 4: Semantic Relation Distribution
The last area of the cost function, not covered by a mutation function, is the semantic relation
distribution. The concept is very similar to the sense distribution mutation. Find the current distribution
for the given semantic relation and shuffle the recessive genes to match the optimal distribution. This
maximizes the cost function by incorporating the proper semantic relation distributions. It also provides
recessive genes a chance to become dominant genes, just as in the sense distribution mutation. The
steps to the semantic relation distribution are below.
1. Compare each average semantic relation value for each sense to the optimal semantic relation
value for that sense. This should result in the number of words that need to change for each
sense.
2. Find the genes that have the lowest gene cost
3. Look at each sense distribution
a. If the current sense distribution has extra words, move the lowest cost genes to a sense
distribution needing words
5.4 Main Genetic Algorithm Function
Earlier the author provided the generic main steps to create a genetic algorithm. The preceding
sections explain how the three main parts (cost function, mating, and mutation) function. However,
none of these sections explain the size of a generation, where the range of values for the cost function
originates, and how chromosomes for the mutation and mating are selected. Details such as this are the
responsibility of the main genetic algorithm function. This function is the center of the genetic
algorithm. The step by step process of this algorithm is below.
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 68
1. Read in the given text, initialize all the WordNet semantic relation information, and assign the
parts of speech with a part of speech tagger
2. Initialize the generation pool. Note that this pool does not keep track of duplicate solutions.
3. Initially find the range of possible scores for each semantic relation using the semantic relation
score mutation. Each semantic relation has an upper and lower boundary. There will be one
process for each boundary. This process is below. The cost function relies on these ranges for
comparing solutions.
a. Start with a solution using the first sense for all words (or the given solution if started
from step 5)
b. Mutate that solution towards the given boundary (maximum or minimum possible
value)
c. Repeat step b until there is a repeat solution or until 25 mutations are performed
d. Report the current semantic relation score as the given boundary.
4. Add all of the solutions from step 3 into the generation pool. On top of that use the weighted
random function from the sampling section to create 25 new solutions.
5. If the generation number is divisible by 10, repeat step 3 using the top solution. Sometimes the
new starting point finds a slightly wider semantic relation range.
6. Mate the top 5 solutions as parent 1 with randomly chosen solutions from the generation pool
as parent 2. After this mate two randomly chosen solutions from the generation pool 20 times.
Add all children to the generation pool.
7. Mutate the top 5 solutions. Then mutate 20 randomly chosen solutions from the generation
pool. Add all new solutions to the generation pool.
8. Reduce the current generation pool by only keeping the top 25 solutions.
9. Repeat steps 5-8 for 25 generations.
5.5 Notes about Speed
Many of the sections above describe complex algorithms containing a large number of
combinations. There are 25 generations with each generation creating 50 new solutions. The cost
function for each solution uses several semantic relations. Each semantic relation is runs on every word
pair combination in a window with 45 possible combinations. The number of windows is the total
number of words in a solution. Every time the cost function ranges change, all of the solutions must
rerun the cost function. The semantic relation score mutation in particular manipulates genes one at a
time, which changes the score of the entire solution. There are a very large number of calculations for
this genetic algorithm.
To deal with the number of calculations, there are several architectural choices and
modifications to the code to reduce the impact of the changes. All of the information is stored in RAM
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 69
to reduce the time it takes to look at the disk, which makes the program run with ~700MB of memory.
The cost function tracks the information in various data structures and minimizes the number of
recalculations mutations and semantic relation range changes cause. All mating and mutation
operations run in parallel to take advantage of multicore processors. However, all of these
improvements are limited. The number of calculations alone causes this program to run for a few
minutes for each SemCor file. Many of the SemEval competitions have more words, so the process will
take longer. This is not a real time solution for large groups of text.
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 70
Chapter 6: Results and Analysis
This project has compared the results within three main areas. The first area focused upon a
comparison against a colleague working on the same problem. The colleague, Michael Billot, has
implemented a page rank algorithm to solve word sense disambiguation. The second area is the
“Genetic Word Sense Disambiguation” study by Zhang (Zhang, Zhou, & Martin, 2008). This paper was
the starting point for this study, and it has significantly influenced the conceptual framework of the
author’s approach. The author has included several changes which are based on reports from other
studies, but their influence has not been as significant. The third area is the SemEval competitions.
Many researchers who did and did not participate in the competitions use these results as a comparison
against other algorithms. All three comparisons should provide significant insight to how well this
algorithm functions.
6.1 Measuring the Results
When the first SemEval competition occurred in 1998, the sponsors sought to compare the
results from all the competitors. They devised three categories for the competition: coverage, recall,
and precision. Coverage is the ratio of how many words the competitor answered to the total number
of words overall. This reflects how much of the solution a competitor answered. Recall is the ratio of
the total number of senses correct within the total number of words provided. This indicates how well a
solution was answered. Precision is the total number of senses correct within the total number of
words answered. This indicates how well an algorithm answers when it does give a sense. SemEval
2007 introduced two different ways to measure a result because some senses in WordNet are too subtle
for even humans to disambiguate. The first method uses the exact sense as a fine word assessment, as
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 71
was done in the previous competitions. The second method combines subtly different senses into one
possible answer as a coarse words assessment. The sponsors evaluated the SemEval 2007 competition
by using the same three numbers separately for both the fine and coarse word assessments.
6.2 Comparison to Michael Billot
Michael Billot is the author’s colleague at the University of Colorado at Colorado Springs. His
solution uses the page rank algorithm to disambiguate verbs. In his study, he used several walkthroughs
for Zelda as the input. He then evaluated the results by hand, a method which presents obvious
difficulties. Unless the verb is clearly a sense, which is rare, it is hard to prove one sense is correct. Even
during the SemEval competitions, the sponsors only have the competitors attempt to find the sense of a
word when over 90% of a group of professional linguists agree. Even if the project uses the same text,
there is no clear indication as to whether the author has chosen the same senses as did Billot, or
whether the sense is correct. With that in mind, comparing against Billot may be a problematic
challenge.
Billot’s page rank algorithm has a 46.4% accuracy according to his study (Billot, 2010).
Considering that he supplies a sense for every verb, this 46.4% figure is the recall and the precision. The
value for the first sense baseline is 77.4%. Since evaluation by hand is problematic at best, this author
did not attempt to disambiguate the walkthrough. However, this author does have the results for
SemCor. For a first sense baseline, the 77.4% baseline is higher than the average baseline, but it is still
within acceptable range. The average coverage for verbs is 99.26%. This is very high, which makes the
recall and precision are very close. The average recall is 49.24% and the average precision is 49.61%.
These percentages suggest that this author’s project evaluates the results more accurately, but the
results are still too close to be certain. The most effective way to tell for sure is for Billot to provide
results for the SemCor files as well.
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 72
6.3 Comparison to Zhang
Zhang’s study, “Genetic Word Sense Disambiguation”, represents the most similar approach to
this project. However, Zhang only evaluates nouns and has not participated in any of the SemEval
competitions. Zhang does, however, provide references to several SemCor files. Overall, Zhang’s
coverage is 100%, so his recall and precision results are the same. This project purposely ignores
pronouns, resulting in average coverage of 89.25%. Within all the SemCor files, Zhang has a
recall/precision of 71.96%. This project has a precision of 62.13% for nouns. Zhang reports a 70%
accuracy on 51 files. This project reports a 70% precision on nine files. This author’s conclusion is that
that this project is not as successful as was Zhang’s. However, one should keep in mind that this
author’s project solved for all words, a significantly more complex study overall.
6.4 SemEval
The SemEval competitions occur every three years. The competitions are devoted to language
processing and typically incorporate an “all words” task. This task requires performing word sense
disambiguation on all the words they have tagged. They then use this information to provide recall,
precision, and coverage for each competing algorithm. Many researchers compare themselves to the
competitors in the SemEval competitions.
6.4.1 SemEval 2
In the SemEval-2 competition in 2001, there were 22 systems participating for the English all
words task (ACL-SIGLEX, 2011). The baseline for using the first sense was 57.0%. The best system had
100% coverage and a 69.8% precision. This author’s project achieved 95.12% coverage and a 52.29%
precision. The comparison shows a 4.71% precision below the baseline and a 17.51% precision below
the best system. The winner was an outlier in this competition. The second place system had a
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 73
precision 5.3% below the first place winner. If this project were competing, it would be in sixth place.
This performance lies somewhere between the middle ranks and the top ranks.
Table 1: The results from SemEval 2
Rank
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
System
SMUaw
CNTS-Antwerp
Sinequa-LIA - HMM
UNED - AW-U2
UNED - AW-U
Michael Hausman
UCLA - gchao2
UCLA - gchao3
CL Research - DIMAP
CL Research - DIMAP (R)
UCLA - gchao
Universiti Sains Malaysia 2
IRST
Universiti Sains Malaysia 1
Universiti Sains Malaysia 3
BCU - ehu-dlist-all
Sheffield
Sussex - sel-ospd
Sussex - sel-ospd-ana
Sussex - sel
IIT 2
IIT 3
IIT 1
Coverage (%)
100
100
100
98.908
98.908
95.12
95.552
95.552
108.5
100
89.729
99.96
47.756
97.897
99.96
50.789
45.37
29.883
31.055
23.332
11.646
11.646
11.646
Recall
0.69
0.636
0.618
0.569
0.55
0.4973
0.454
0.453
0.451
0.451
0.449
0.36
0.357
0.338
0.336
0.291
0.2
0.169
0.169
0.14
0.038
0.034
0.033
Precision
0.69
0.636
0.618
0.575
0.556
0.5229
0.475
0.474
0.416
0.451
0.5
0.36
0.748
0.345
0.336
0.572
0.44
0.566
0.545
0.598
0.328
0.294
0.287
6.4.2 SemEval 3
In the SemEval-3 competition of 2004, 26 systems participated in the all words task (Snyder &
Palmer, 2004). The baseline of using the first sense was 60.9% or 62.4% depending on the treatment of
compound words. This project used compound words; 62.4% was the baseline. The best system had
100% coverage and a 65.1% precision. This project had 96.93% coverage and a 53.79% precision, 8.61%
below the baseline and 11.31% below the best system. These results are disappointing considering that
the average SemCor precision is close to the top performing system. It should be noted, however, that
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 74
this this author’s work would rank 15th in the competition. This is in the middle ranks of the
competition.
Table 2: The results from SemEval 3
Rank
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
System
GAMBL-AW-S
SenseLearner-S
Koc University-S
R2D2: English-all-words
Meaning-allwords-S
Meaning-simple-S
LCCaw
upv-shmm-eaw-S
UJAEN-S
IRST-DDD-00-U
University of Sussex-Prob5
University of Sussex-Prob4
University of Sussex-Prob3
DFA-Unsup-AW-U
Michael Hausman
KUNLP-Eng-All-U
IRST-DDD-LSI-U
upv-unige-CIAOSENSO-eaw-U
merl.system3
upv-unige-CIAOSENSO2-eaw-U
merl.system1
IRST-DDD-09-U
autoPS-U
clr04-aw
autoPSNVs-U
merl.system2
DLSI-UA-all-Nosu
Coverage (%)
100
98.62
98.61
100
99.68
99.84
98.70
98.21
97.84
99.83
97.09
95.65
95.46
98.03
96.93
97.25
75.04
82.62
97.64
74.18
97.39
60.49
88.37
85.18
62.88
73.33
80.17
Recall
0.651
0.642
0.639
0.626
0.623
0.61
0.606
0.605
0.588
0.582
0.568
0.55
0.547
0.546
0.5213
0.496
0.496
0.48
0.456
0.451
0.447
0.441
0.433
0.431
0.354
0.352
0.275
Precision
0.651
0.651
0.648
0.626
0.625
0.611
0.614
0.616
0.601
0.583
0.585
0.575
0.573
0.557
0.5379
0.51
0.661
0.581
0.467
0.608
0.459
0.729
0.49
0.506
0.563
0.48
0.343
6.4.3 SemEval 2007
In the SemEval 2007 competition, there were 14 systems participating in the competition (15 if
the task organizers system is included) (Navigli, 2007). In this case, all the results are for the coarse
words evaluation. The baseline using the first sense has a 78.89% precision. The best participating
system has 100% coverage and an 82.5% precision (the organizer’s system has 100% coverage and an
83.21% precision). This project has 100% coverage and a 74.51% precision. This is 4.38% below the
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 75
baseline and 8.34% below the organizer’s system. This definitely does not perform as well as the top
system. If this project participated in the competition, it would have ranked in 7th place. This is the
center rank.
Table 3: The coarse results from SemEval 2007
Rank
1
2
3
4
5
6
7
8
9
10
11
12
13
14
System
NUS-PT
NUS-ML
LCC-WSD
GPLSI
BLMFS
UPV-WSD
Michael Hausman
TKB-UO
PU-BCD
RACAI-SYNWSD
SUSSX-FR
USYD
UOFL
SUSSX-C-WD
SUSSX-CR
Coverage (%)
100
100
100
100
100
100
100
100
90.1
100
72.8
95.3
92.7
72.8
72.8
Recall (%)
82.5
81.58
81.45
79.55
78.89
78.63
74.51
70.21
62.8
65.71
52.23
56.02
48.74
39.71
39.53
Precision (%)
82.5
81.58
81.45
79.55
78.89
78.63
74.51
70.21
69.72
65.71
71.73
58.79
52.59
54.54
54.3
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 76
Chapter 7: Conclusions and Future Research
This project has attempted to solve for word sense disambiguation. Solving word sense
disambiguation would allow several applications, like language translation, to work much more
accurately. To accomplish this, the author investigated several semantic relations and transformed
them into an optimization problem. This way a genetic algorithm could solve the problem. The genetic
algorithm used a dominant gene technique to converge on an answer. The author then compared the
results to several other researchers, including the competitors in the SemEval competitions. If this
algorithm had competed in the SemEval competitions, it would have rank in the middle. This means
that it is not the optimal algorithm, but it still works well enough to be of interest since it is not in the
lower ranks.
7.1 Algorithm Weaknesses
One of the reasons why the results are not 100% has to do with the available semantic relations.
An in-depth investigation of some of the words reveals that there is simply not enough information
provided in context. Some words have a correct sense of something other than the most common
sense, yet frequency is the only semantic relation that applies to the word. Frequency directs the result
to the incorrect sense. False positives and misleading information do have an effect, but a large
proportion of the words have insufficient information in order to provide acceptable results. In most of
these cases, the information supplied by WordNet will not help. This includes several other semantic
relations that were not part of this project. The author cursorily explored Lesk, but upon reflection it
proved unhelpful. It is significant that many other algorithms appear to have similar problems. One of
the top competitors in the third SemEval competition, Sense Learner, openly admits that they do have
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 77
any information on 15% of the words (Mihalcea & Csomai, 2005). These words simply chose the most
frequent definition as a default. Regardless, something needs to account for these words.
This algorithm does appear to work fairly well with the available WordNet information. Many
other researchers assume that the highest scores work while this algorithm attempts to adjust for the
weaknesses within each semantic relation. However, there still are possible improvements around
semantic relations in the cost function. One of the more significant problems is how large the 90%
window is for some semantic relations. If there were a way to dynamically adjust the maximum point
and reduce the deviation, the answers would improve. At this writing, both the distributions and the
relations rely on averages. The distributions require more investigation and adjustments. Perhaps the
distributions would function more effectively if they accounted for the various parts of speech
individually. One can only surmise in this regard, but it seems logical that additional percentage points
make a significant difference in the SemEval competitions.
7.2 Future Possibilities
There were two tool changes mentioned earlier that may foreshadow future research. The first
was importing the “WordNet Lexical Markup Framework” from the SemEval 2010 competition. This
would allow the author’s project to compare the SemEval 2010 results and to look at results in different
languages. Since one possible application of word sense disambiguation is machine text translation, the
performance in other languages would be very helpful to know. The second change is continuing
investigation of OntoNotes. The semantic and lexical relations will be different for OntoNotes. If the
plans for exploring OntoNotes take place, the research results will be more useful than those relations in
WordNet. At the very least, the algorithm will be able to compete in SemEval in the future if the
competition replaces WordNet. These two changes would help in the future and may provide insight for
other changes to this algorithm.
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 78
The top-rated algorithms in the SemEval competition typically account for a wider range of
information. This algorithm uses semantic relation information from WordNet. Their algorithms literally
look at everything they could think of. This includes information such as: A) “Is this sense typically the
end of a sentence?” or B) “Is this word followed by a noun?” and C) “Does this word end in ‘ing’ all the
time?” This information feeds into various types of algorithms many of which are extremely
complicated. Outstanding competitors have used multiple information sources. One option that may
help this algorithm is the use of more than WordNet semantic relations.
A final option is exploration of punctuation. For example, examine the statement, “Let’s go eat,
grandma!” If no comma separated eat and Grandma, the sentence would take on a most horrific
meaning. Sentence sense changes with punctuation. This author suggests that exploration of the field
of punctuation research in computer systems language development studies may grow into a field that
significantly changes the focus of the discipline and provides data to scaffold word sense disambiguation
on a most significant scale.
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 79
Appendix
The following sections contain several of the detailed results developed throughout this project.
This includes a short table of the types of SemCor files, the proportion statistics for Cost Method 3, the
distributions for Cost Method 4, the distributions for Cost Method 5, and results for every SemCor file.
Appendix A: SemCor Files
Each SemCor file has a similar name: br-*##. The “br” is short for the brown corpus. The “*”
represents a letter that indicates what genre of text is tagged. The “##” represents a number that
separates different files in a genre.
Table 4: The SemCor file letter indicating type of resource the original text came from.
Informative Prose (374 samples)
Letter Genre
Press: Reportage
A
Press: Editorial
B
Press: Reviews
C
Religion
D
Skill and Hobbies
E
Popular Love
F
Belles Letters, Biography, Memoirs, etc.
G
Miscellaneous
H
Learned
J
Imaginative Prose (126 samples)
Letter Genre
K
General Fiction
L
Mystery and Detective Fiction
M
Science Fiction
N
Adventure and Western Fiction
P
Romance and Love Story
R
Humor
Table 5: The various SemCor files
File
A01
A02
A03
A04
A05
A06
A07
A08
A09
A10
A11
A12
Author
Atlanta Constitution
Dallas Morning News
Chicago Tribune
Christian Science Monitor
Providence Journal
Newark Evening News
New York Times
Times-Picayune, New Orleans
Philadelphia Inquirer
Oregonian, Portland
Sun, Baltimore
Dallas Morning News
Source
Political Reportage
Political Reportage
Political Reportage
Political Reportage
Political Reportage
Political Reportage
Political Reportage
Political Reportage
Political Reportage
Political Reportage
Sports Reportage
Sports Reportage
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 80
File
A13
A14
A15
A16
A17
A18
A19
A20
A21
A22
A23
A24
A25
A26
A27
A28
A29
A30
A31
A32
A33
A34
A35
A36
A37
A38
A39
A40
A41
A42
A43A
A43B
A44A
A44B
B01
B02
B03
B04
B05
B06
B07
B08
B09
Author
Rocky Mountain News
New York Times
St. Louis Post-Dispatch
Chicago Tribune
Rocky Mountain News
Philadelphia Inquirer
Sun, Baltimore
Chicago Tribune
Detroit News
Atlanta Constitution
Oregonian, Portland
Providence Journal
San Francisco Chronicle
Dallas Morning News
Los Angeles Times
Wall Street Journal
Dallas Morning News
Los Angeles Times
Miami Herald
San Francisco Chronicle
Washington Post
New York Times
James J. Maguire
William Gomberg
Time
Sports Illustrated
Newsweek
Time
Robert Wallace
Newsweek
U. S. News & World Report
U. S. News & World Report
John Tebbel
Gilbert Chapman
Atlanta Constitution
Christian Science Monitor
Detroit News
Miami Herald
Newark Evening News
St. Louis Post-Dispatch
New York Times
Atlanta Constitution
Christian Science Monitor
Source
Sports Reportage
Sports Reportage.
Sports Reportage
Society Reportage
Society Reportage
Society Reportage
Spot News
Spot News
Spot News
Spot News
Spot News
Spot News
Spot News
Financial Reportage
Financial Reportage
Financial Reportage
Cultural Reportage.
Cultural
Cultural Reportage
Cultural Reportage
Cultural Reportage
News of the Week in Review
A Family Affair
Unions and the Anti-Trust Laws
National Affairs
A Duel Golfers Will Never Forget
Sports
People. Art & Education
This Is The Way It Came About
National Affairs
Better Times for Turnpikes
A Plan to Free U. S. Gold Supply
Books Go Co-operative
Reading and the Free Society
Editorials
Editorials
Editorials
Editorials
Editorials
Editorials
Editorials
Columns
Columns
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 81
File
B10
B11
B12
B13
B14
B15
B16
B17
B18
B19
B20
B21A
B21B
B22
B23A
B23B
B24
B25A
B25B
B26
B27
C01
C02
C03
C04
C05
C06
C07
C08
C09
C10
C11
C12
C13
C14
C15
C16
C17
D01
D02
D03
D04
D05
Author
Sun. Baltimore
Los Angeles Times
Newark Evening News
Times-Picayune, New Orleans
Atlanta Constitution
Providence Journal
Chicago Tribune
Newark Evening News
New York Times
Philadelphia Inquirer
Nation
Gerald W. Johnson
James Deakin
Commonweal
William F. Buckley, Jr.
James Burnham
Time
Alexander Werth
Peter Solsich, Jr.
National Review
Saturday Review
Chicago Daily Tribune
Christian Science Monitor
New York Times
Providence Journal
Christian Science Monitor
Wall Street Journals
New York Times
Providence Journal
New York Times
Providence Journal
New York Times
Christian Science Monitor
Wall Street Journal
New York Times
Life
Saturday Review
Time
William Pollard
Schubert Ogden
Edward E. Kelly
Jaroslav Pelikan
Perry Miller
Source
Columns
Columns
Columns
Columns
Columns
Letters to the Editor
Voice of the People
What Readers Have to Say
Letters to the Times
The Voice of the People
Editorials
The Cult of the Motor Car
How Much Fallout Can We Take
Week by Week
We Shall Return
Tangle in Katanga
Reviews
Walkout in Moscow
The Armed Superpatriots
To the Editor
Letters to the Editor
Reviews
Reviews
Reviews
Reviews
Reviews
Reviews
Reviews
Reviews
Reviews
Reviews
Reviews
Reviews
Reviews
Reviews
Reviews
Reviews
Reviews
Physicist and Christian
Christ Without Myth
Christian Unity in England
The Shape of Death
Theodore Parker: Apostasy With in Liberalism
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 82
File
D06
D06B
D06C
D07
D08
D09
D10
D11
D12
D13A
D13B
D14
D15
D16A
D16B
D17A
D17B
E01A
E0lB
E02A
E02B
E03
E04A
E04B
E04C
E05A
E05B
E05C
E06
E07
E08
E09A
E09B
E10
E11
E12
E13
E14
E15A
E15B
E16
E17A
Author
A Howard Kelly
Shirley Schuyler
Nathanael Olson
Peter Eldersveld
Schuyler Cammann
Eugene E. Golay
Huston Smith
Paul Ramsey
Kenneth Underwood and Widen
Jacobson
Donald H. Andrews
George Bo Longstreet
Kenneth S. Latourette
Ernest Becker
Anonymous
Harold Brenneman
Anonymous
J. I. Rivero
Ben Welder
Joe Welder
Dorothy Schroeder
Anonymous
D. F. Martin
Harris Goldsmith
Robert C. Marsh
R.D.D.
Paul Nigro
Patricia Barney
Anonymous
Joseph E. Choate
Paul Larson and Gordon Odegard
Don Francisco
Don McMahan
Edith Shaw
Larry Koller
Idwal Jones
Julia Newman
Robert Deardorff
Ann Carnahan
Anonymous
Anonymous
Hal Kelly
Anonymous
Source
Out of Doubt into Faith
Not as the World Giveth
Are You in Orbit?
Faith Amid Fear
The Magic Square of Three
Organizing the Local Church
Interfaith Communication: The Contemporary Scene
War & the Christian Conscience
Probing the Ethics of Realtors
The New Science & the New Faith
The Seeming Impossible
Christianity in a Revolutionary Age
Zen: A Rational critique
What the Holy Catholic Bible Teach
Notice You May Do As You Please
Guideposts: 15th Anniversary Issue
The Night Our Paper Died
Henri de Courcy: Jr. Mr. Canada
The Mark of the Champion
Plant a Carpet of Bloom
Avocado is Something Special
Will Aircraft or Missiles Win Wars?
The Schnabel Pro Arte Trout
The True Sound of a Solid Second
Review of Adam, Giselle
The Younger Generation
Use of Common Sense Makes Dogs Acceptable
The Malady Lingers On
The American Boating Scene
How to Design Your Interlocking Frame
Formulas and Math Every Hot Rodder Should Know
The Week at Ben White Raceway
The Picture at Del Mar
The New Guns of 61
Santa Cruz Run
Travel and Camera USA
Step by Step through Istanbul
Nick Manero's Cook-out Barbecue Book
Pottery from Old Molds
Knitting Knacks
Build Hotei
This is the Vacation Cottage You Can Build
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 83
File
E17B
E18A
E18B
E19
Source
Care and Basic Use of the Drill Press
The Bridge Over the Merrimac
Veteran Philippi Bridge
How to Own a Pool and Like It.
E20
E21
E22A
E22B
E23
E24
E25
E26
E27A
E27B
E28
E29
E30
E31A
E31B
E32
E33A
E33B
E34
E35
E36
F01
Author
Patrick K. Snook
Lura W. Watkins
Boyd B. Stutler
Booth Hemingway and Stuart H.
Brown
Anonymous
Richard McCosh
Roy Harris
Helen Havener
Norman Kent
Bonnie Prudden
Walter Ho Buchsbaum
Bern Dibner
Mike Bay
James S. Boyd
John R. Sargent
Edward A. Walton
Anonymous
Jim Dee
George Laycock
E. J. Tangerman
Robert Gray
Chet Cunningham
Anonymous
Anonymous
Ethel Norling
Rosemary Blackmon
F02
F03
F04
F05
F06A
F06B
F07A
F07B
F08
F09A
F09B
F10
F11
F12
F13
Glenn Infield
Nathan Rapport
Ruth F. Rosevear
Richard S. Allen
Alice Ho Austin
Harold P. Winchester
Marvin Sentnor and Stephen Hult
Ho Walter Yoder
Philip Reaves
David Martinson
Isel D. Rugget
Jack Kaplan
Lillian Pompian
Marian Neater
Orlin J. Scoville
America's Secret Poison Gas Tragedy
I've Been Here Before
North Country School Cares for the Whole Child
When Fogg Flew the Mail
Let's Discuss Retirement
What It Means to be Creative
How to Have a Successful Honeymoon
Attitudes Toward Nudity
Who Rules the Marriage Bed?
Fantastic Life & Death of the Golden Prostitute.
When It Comes to Carpets
Therapy by Witchcraft
Tooth-Straightening Today
New Methods of Parapsychology.
Part-time Farming
What You Should Know About Air Conditioning
Recreation Site Selection
Roy Harris Salutes Serge Prokofieff
A 30 Years War
The Watercolor Art of Roy M. Mason
The Dancer & the Gymnast
Advances in Medical Electronics
Oersted & the Discovery of Electromagnetism
What Can Additives Do for Ruminants?
Which Feed Bunk for You
Where to Aim Your Planning
On Education for the Interior Designer
The Attack on Employee Services
Expanding Horizons
The Challenge
Which Way Up. Technical or Management?
Fifty Houses, One Tank
Truck Talk
The New Look in Signs
The Industrial Revolution in Housing
Renting a Car in Europe
How Much Do You Tell When You Talk?
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 84
File
F14
F15
F16
F17
F18
Fl9
F20
F21A
F21B
F21C
F22
F23
F24
F25
F26
F27
F28
F29
F30
F31
F32
F33
F34
F35
F36
F37
F38
F39
F40
F41
F42
F43
F44
F45
F46
F47
F48
G01
G02
G03
G04
Author
Harold Rosenberg
John A. O'Brien
James Boylan
John Harnsberger and Robert P.
Wilkins
Bell I. Willy
Tristram P. Coffin
Kenneth Allsop
Joseph Bernstein
L. Don Leet
L. Don Leet
Booton Herndon
Barry Goldwater
Peter J. White
David Boroff
Amy Lathrop
Creighton Churchill
Frank O. Gatell
Douglass Cater
Frederic A Birmingham
Edward Do Radin
Florence M. Read
James Be Conant
Frederic R. Senti and W. Dayton
Maclay
Ramon F. Adams
Robert Easton and Mackenzie Brown
Samuel M. Cavert
Robert Smith
Clark E. Vincent
William Greenleaf
George W. Oakes
James Baldwin
Frank Getlein and Harold C. Gardiner
Gibson Winter
Paul C. Phillips
Russell Baker
Clara L. Simerville
Paul Ramsey
Edward P. Lawton
Arthur S. Miller
Peter Wyden
Eugene Burdick
Source
The Trial and Eichmann
Let's Take Birth Control Out of Politics
Mutiny
Transportation on the Northern Plains
Home Letters of Johnny Reb & Billy Yank
Folklore in the American Twentieth Century
The Bootleggers and Their Era
Giant Waves
Introduction
The Restless Earth and Its Interiors
From Custer to Korea, the 7th Cavalry
A Foreign Policy for America
Report on Laos
Jewish Teen-Age Culture
Pioneer Remedies from Western Kansas
A Notebook for the Wines of France
Doctor Palfrey Frees His Slaves
The Kennedy Look in the Arts
The Ivy League Today
Lizzie Borden: The Untold Story
The Story of Spelman College
Slurs and Suburbs
Age-old uses of Seeds and Some New Ones
The Old-time Cowhand
Lord of Beasts
On the Road to Christian Unity
Baseball in America
Unmarried mothers
Monopoly on Wheels
Turn Right at the Fountain
Nobody Knows My Name
Movies, Morals, and Art
The Suburban Captivity of the Churches
The Fur Trade
An American in Washington
Home Visits Abroad
Christian Ethics & the Sit-In
Northern Liberals & Southern Bourbons
Toward a Concept of National Responsibility
The Chances of Accidental War
The Invisible Aborigine
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 85
File
G05
G06A
G06B
G07
G08
G09
G10
G11
G12
G13
G14
G15
G16
G17
G18
Gl9
G20
G21
G22
G23
G24
G25
G26
G27
G28
G29
G30
G31
G32
G33
G34
G35
G36
G37
G38
G39
G40
G41
G42
G43
G44
G45
Author
Terence O'Donnell
Ruth Berges
Henry W. Koller
Richard B. Morris
Frank Murphy
Selma J. Cohen
Clarence Streit
Frank Oppenheimer
Tom F. Driver
Charles Glicksberg
Helen H. Santmeyer
Howard Nemerov
John F. Hayward
Randall Stewart
Charles W. Stork
R. F. Shaw
Dan McLachlan, Jr.
Brainard Cheney
Kenneth Reiner
William C. Smith
Sanchia Thayer
Stanley Parry
Selma Fraiberg
Matthew Josephson
Arlin Turner
Anonymous
Norwood R. Hanson
Irving Fineman
Finis Farr
Virgilia Peterson
Harry Golden
Dwight D. Eisenhower
DeWitt Copp & Marshall Peck
Gordon L. Hall
Bertrand A. Goldgar
Edward Jablonski
Gene Fowler
Lillian R. Parka and Frances S.
Leighton
Harold D. Lasswell
Robert E. Lane
Newton Stallknecht
W. A. Swanberg
Source
Evenings at the Bridge
William Steinberg, Pittsburgh's Dynamic Conductor
German Youth Looks to the Future
Seven Who Set Our Destiny
New Southern Fiction: Urban or Agrarian?
Avant-Garde Choreography
How the Civil War Kept You Sovereign
Science and Fear
Beckett by the Madeleine
Sex in Contemporary Literature
There Were Fences
Themes and Methods: Early Storie of Thomas Mann
Mimesis & Symbol in the Arts
A Little History, a Little Honesty
Verner von Heidenstam
The Private Eye
Communication Networks & Monitoring
Christianity & the Tragic Vision
Coping with Runaway Technology
Why Fear Ideas
Personality & Moral Leadership
The Restoration of Tradition
Two Modern Incest Heroes
Jean Hélion. The Return from Abstract Art
William Faulkner, Southern Novelist
References for the Good Society
Copernican & Keplerian Astronomy
Woman of Valor: Life of Henrietta Szold 1860-1945
Frank Lloyd Wright
A Matter of Life and Death
Carl Sandburg
Peace With Justice
Betrayal at the UN
Golden Boats from Burma
The Curse of Party
Harold Arlen Happy with the Blues
Skyline: A Reporter's Reminiscences of the 1920s.
My Thirty Years Backstairs at the White House
Epilogue
The Liberties of Wit
Ideas and Literature
Citizen Hearst: A Biography of W. R. Hearst
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 86
File
G46
G47
G48
G49
G50
G51
G52
G53
G54
G55
G56
G57
G58
G59
G60
G61
G62
G63
G64
G65
G66
G67
G68
G69
G70
G71
G72
G73
G74
G75
H01
Author
Henry R. Winkler
Carry Davis
Francis F. McKinney
Paul van K. Thomson
Curtis C. Davis
Ilka Chase
Robert L. Duncan
Bertram Lippincott
Mabel W. Wheaton & LeGette Blythe
Ralph E. Flanders
Keith F. McKean
Robin M. Williams, Jr.
North Callahan
Esther R. Clifford
Gertrude Berg & Cherney Berg
Donald A. White
C. H. Cramer
George Steiner
Mark Eccles
Timothy P. Donovan
Van Wyck Brooks
Mark Schorer
Harris F. Fletcher
Mark R. Hillegas
Joseph W. Krutch
Joseph Frank
J W. Fulbright
Carolyn See
John McCormick
George Garrett
U. S Dep't of Commerce
Source
George Macaulay Trevelyan
The World Is My Country
Education in Violence
Francis Thompson, a Critical Biography
The King's Chevalier
The Carthaginian Rose
Reluctant General
Indians, Privateers, and High Society
Thomas Wolfe & His Family
Senator from Vermont,. 112
The Moral Measure of Literature
Values & Modern Education in the United States
Daniel Morgan
A Knight of Great Renown
Molly and Me
Litus Saxonicum
Newton D. Baker
The Death of Tragedy
Shakespeare in Warwickshire
Henry Adams & Brooks Adams
From the Shadow of the Mountain
Sinclair Lewis: An American Life
The Intellectual Development of John Milton
Dystopian Science Fiction
If You Don't Mind My Saying So
André Malraux: The Image of Man
For a Concert of Free
The Jazz Musician as Patchen's Hero
The Confessions of Jean Jacques Krim
A Wreath for Garibaldi
Handbook of Federal Aids to Communities
H02
H03
H04
H05
H06
H07
H08
H09
H10
H11
H12
U. S. Dep't of State
U. S. 87th Congress
R. I. Legislative Council
R. I. Leglelative Council
R. I. Development Council
R. I. Legislative Council
John A. Notte, Jr.
U. S. 87th Congress
U. S. Dep't of Defense
U. S. Dep't of Commerce
U. S. 87th Congress
An Act for International Development
House Document No. 487
State Automobiles & Travel Allowances
Taxing of Movable Tangible Property
Annual Report, 1960
linlform Fiscal Year for Municipalities
R. I. Governor's Proclamations
Public Laws 295, 300, 347
Medicine in National Defense
1961 Reaearch Highlights, Nat'1 Bureau of Standards
Legislation on Foreign Rels
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 87
File
H13
Author
U. S. 87th Congreas
H14
H16
U. S. Dep't of Health, Education &
Welfare
U. S. Office of Civil and Defence
Mobilization
U. S. Reports
H17
U. S. Reports
H18
Hl9
H20
H21
H22
H23
H24
H25
H26
H27A
H27B
Dean Rusk
Peace Corps
U. S. Dep't of Agriculture
Dwight D. Eisenhower
U. S. Dep't of State
U. S. Federal Communications
Commiasion
U. S. Tresaury Dep't
Guggenheim Foundation
Anonymous
Robert Leeson
Leesona Corporation
H28
H29
H30
J01
Carleton College
Sprague Electric Company
Carnegie Foundation
Cornell H. Mayer
Your Federal Income Tax
Report of the Secretary Gen'1
A Brief Background of Brown & Sharpe
Leesona Corporation President's Report
More Efficient Production for Expanding Textile
Markets
Carleton College Bulletin
Sprague Log
Annual Report of Year Ending June 30, 1961
Radio. Emission of the Moon and Planets
J02
J03
J04
J05
J06
J07
J08
J09
J10
J11
J12
J13
J14
J15
Jl6
J17
R. C. Binder et al.
Harry H. Hull
James A. Ibers et al.
John R. Van Wazer, ed.
Francis J. Johnston & John E. Willard
J. F. Vedder
LeRoy Fothergill
M. Yokayama et al
B. J. D. Meeuse
Clifford H Pope
Richard F McLaughlin et al.
S. Idell Pyle et al.
Jacob Robbins et al.
J. W. C. Hagstrom et. al.
A. N. Nagaraj & L. M. Black
E. Gellhorn
1961 Heat Transfer & Fluid Mechanics Institute
Normal Forces & Their Thermodynamic Significance
Proton Magnetic Resonance Study
Phosphorus and Its Compounds
Exchange Reaction Between C12 and CC14
Micrometeorites
Biological Warfare
Chemical & Serological Characteristics
The Story of Pollination
The Ciant Snakes
A Study of the Subgross Pulmonary Anatomy
Onsets, Completions & Spans
The Thyroid-Stimulating Hormone
Debilitating Muscular Weakness
Localization of Wound-Tumor Virus Antigen
Prolegomena to a Theory of the Emotions
H15
Source
Congressional Record: Extension of Remarks. May 2,
1961
Grants-in-Aid and Other Financial Assistance Programs
The Family Fallout Shelter
Cases AdJudged in the Supreme Court, October Tenm
1960
Cases AdJudged in the Supreme Court, October Tenm
1959-60
The Department of State
Fact Book
Development Program for the National Forests
Public Papers, 1960-61
U. S. Treatiea and Other International Agreements
Pederal Communications Commission Reports
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 88
File
J18
Jl9
J20
J21
J22
J23
J24
J25
J26
J27
J28
J29
J30
J31
J32
J33
J34
J35
J36
J37
J38
J39
J40
J41
J42
J43
J44
J45
J46
J47
J48
J49
J50
J51
J52
J53
J54
J55
J56
J57
Author
Kenneth Hoffman & Ray Kunze
Frederick Mosteller et al.
R. P. Jerrard
C. R. Wylie, Jr.
Max F. Millikan & Donald L.
Blackmer, editors
Joyce O. Hertzler
Howard J. Parad
Sister Claire M. Sawyer
Frank Lorimer
Dale L. Womble
William H. Ittelson & Samuel B.
Kutash, editors
Jesse W. Grimes & Wesley Allinsmith
Raymond J. Corsini
Harold Searles
Hugh Kelly & Ted Ziehe
Ralph Bc Long
H.A. Cleason
A. T. Kroeber
D. F. Fleming
Douglas Ashford
Committee for Economic
Development
William O'Connor
James J. O'Leary
Allan J. Braff & Roger F. Miller
Morton A. Kaplan ~ Nicholas
Katzenbach
Wallace Mendelson
J. Mitchell Reese, Jr,
Albert Schreiber et al.
Irving Perluss
William S. Ragan
Paul Cooke
Robert J. Havighurst
James C. Bonbright
Irving L. Horowitz
Brand Blanshard
William S. Haymond
Chester G. Starr
Jim B. Pearson
Edwin L. Bigelow & Nancy H. Otis
J. H. Hexter
Source
Linear Algebra
Probability with Statistical Applications
Inscribed Squares in Plane Curves
Line Involutions in S3
The Emerging Nations
American Social Institutions
Preventive Casework: Problems & Implications
Some Aspects of Fertility of a Tri-Racial Isolate
Demographic Information on Tropical Africa
Functional Marriage Course for the Already Married
Perceptual Changes in Psychopathology
Compulsivity, Anxiety & School Achievement
Roleplaying in Business & Industry
Schizophrenic Communication
Glossary Lookup Made Easy
The Sentence & Tts Parts
Review of African Language Studies
Semantic Contribution of Lexicostatistics
The Cold War & Its Origins
Elections' in Morocco: Progress or Confusion
Distressed Areas in a Growing Economy
Stocks, Wheat & Pharaohs
The outlook for Interest Rates in 1961
Wage-Price Policies Under Public Pressure
The Political Foundation of Internationa1 Law
Justices Black & Frankfurter
Reorganization Transfers
Defense Procurement & Small Business
Agricultural Labor Disputes in California 1960
Teaching America's Children.
Desegregated Education in the Middle-South Region
Social-Class Influences on American Education
Principles of Public Utility Rates
Philosophy, Science & the Sociology of Knowledge
The Emotive Theory
Is Distance an Original Factor in Vision?
The Origins of Greek Civilization 1100-650 B. C
The Maxwell Land Grant
Manchester, Vermont, A Pleasant Land
Thomas More: on the Margins of Modernity
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 89
File
J58
J59
J60
J61
J62
J63
J64
J65
J66
J67
J68
J69
J70
J71
J72
J73
J74
J75
J76
J77
Author
John M, Ray
Clement Greenberg
Robert A. Futterman
Allyn Cox
Jimmy Ernst
John H. Schaar
Katherine G. McDonald
Samuel Hynes
Kenneth Rexroth
William Whallon
Charles R. Forker
I. B. M. Corporation
Ross E. McKinney & Howard Edde
Thomas D. McGrath
Mellon Institute
Nat'l Research Council
Harlan W. Nelson
W. K. Asbeck
Joel Frados, editor
William D. Appel, editor
J78
J79
J80
K01
K02
K03
K04
K05
K06
K07
K08
K09
K10
K11
K12
Kl3
K14
K15
K16
Kl7
K18
K19
Paul J. Dolon & Wilfrid F. Niklas
Rutherford Aris
C. J. Savant Jr. & R. C. Howard
Christopher Davis
Clayton C. Barbeau
Tristram Coffin
W. E. B. Du Bois
David Stacton
Louis Zara
Francis Pollini
Guy Endore
Howard Fast
Gladys H. Barr
Robert Penn Warren
Gerald Green
William Maxwell
Irving Stone
Ann Hebson
Stephen Longstreet
Leon Uris
John Dos Passos
Robert J Duncan
Source
Rhode Island's Reactions to John Brown's Raid
Collage
The Future of Our Cities
Completing & Restoring the Capitol Frescos
A Letter to Artists of the Soviet Union
Escape from Authority, Perspectives of Erich Fromm
Figures of Rebellion
The Pattern of Hardy's Poetry
Disengagament: The Art of the Beat Generation
The Diction of Beowulf
The Language of Hands in Great Expectations
IBM 7070, Autocoder Reference Manual
Aerated Lagoon for Suburban Sewage Disposal
Submarine Defense
Annual Report; 1960, Independent Research
Directory of Continuing Numerical Data Projects
Food Preservation by Ionizing Radiation
Forces in Coatings Removal Knife Cutting Method
Survey of Foamed Plastics
1961 Technical Manual of American Ass'n of Textile
Chemists & Colorists
Gain & Resolution of Fiber Optic Intensifier
The O'ptim.A1 Design of Chemical Reactors
Principles of Inertial Navigation
First Family
The Ikon
Not to the Swift
Worlds of Color
The Judges of the Secret Court
Dark Rider
Night
Voltaire! Voltaire!
April Morning
The Master of & Geneva
Wilderness
The Heartless Light
The Chateau
The Agony & the Ecstasy
The Lattimer Legend
Eagles Where I Walk
Mila 8
Midcentury
The Voice of Strangers
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 90
File
K20
K21
K22
K23
K24
K25
K26
K27
K28
K29
L01
L02
L03
L04
L05
L06
L07
L08
L09
L10
L11
L12
L13
Ll4
L15
L16
L17
L18
L19
L20
L21
L22
L23
L24
M01
M02
M03
M04
M05
M06
N0l
N02
N03
Author
Guy Bolton
Bruce Palmer
John Cheever
Prieda Arkin
W. H. Gass
Arthur Miller
Jane G. Rushing
E. Lucas Myers
Sallie Bingham
Marvin Schiller
Winfred Van Atta
A. A. Fair
Amber Dean
David Alexander
Brett Halliday
Thomas B. Dewey
Genevieve Golden
Dell Shannon
Mignon G. Eberhart
Harry Bleaker
Hampton Stone
Whit Masterson
Dolores Hitchens
Frances & Richard Lockridge
Doris M. Disney
Alex Gordon
Brent James
George H. Coxe
Brad Williams
Ed Lacy
Helen McCloy
S. L. M. Barlow
J. W. Rose
Fredric Brown
Robert Heinlein
Philip J. Farmer
James Blish
Jim Harmon
Anne McCaffrey
Cordwainer Smith
Wayne D. Overholser
Clifford Irving
Cliff Farrell
Source
The Olympians
My Brother's Keeper
The Brigadier & the Golf Widow
The Tight of the Sea
The Pedersen Kid
The Prophecy
Against the Moon
The Vindication of Dr. Nestor
Moving Day
The Sheep's in the Meadow.
Shock Treatment
Bachelors Get Lonely
Encounter With Evil
Bloodstain
The Careless Corpse
Hunter at Large
Deadlier Than the Male
The Ace of Spades
The Cup, the Blade or the Swords
Impact
The Man Who Looked Death in the Eye
Evil Come, Evil Go
Footsteps in the Night
Murder Has Its Points
Mrs. Meeker's Money
The Cipher
Night of the Kill
Error of Judgment
Make a Killing
Death by the Numbers
The Black Disk
Monologue of Murder
Try My Sample Murders
The Murders
Stranger in a Strange Land
The Lovers
The Star Dwellers
The Planet with No Nightmare
The Ship Who Sang
A Planet Named Shayol
The Killer Marshall
The Valley
The Trail of the Tattered Star
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 91
File
N04
N05
N06
N07
N08
N09
N10
N11
N12
N13
N14
N15
N16
N17
N18
Nl9
N20
N21
N22
N23
N24
N25
N26
N27
N28
N29
P01
P02
P03
P04
P05
P06
P07
P08
P09
P10
P11
P12
P13
P14
P15
P16
P17
Author
James D. Horan
Richard Ferber
Thomas Anderson
Todhunter Ballard
Mary Savage
Jim Thompson
Joseph Chadwick
Gene Caesar
Edwin Booth
Martha F. McKeown
Peter Field
Donald J. Plantz
Ralph J. Salisbury
Richard S. Prather
Peter Bains
David Jackson
T. C. McClary
C. T. Sommers
Gordon Johnson
Wheeler Hall
T. K. Brown III
Wesley Newton
Paul Brock
James Hines & James Morris
Ralph Grimshaw
Harlan Ellison
Octavia Waldo
Ann Ritner
Clark McMeekin
B. J. Chute
Allan R. Bosworth
Richard Tiernan
Vina Delmar
R. Leslie Course
Jesse Hill Ford
Jay Williams
Bessie Breuer
Morley Callaghan
Frank B. Hanes
Livingston Biddle, Jr.
Loretta Burrough
Margery F. Brown
Al Hine
Source
The Shadow Catcher
Bitter Valley
Here Comes Pete Now
The Night Riders
Just for Tonight
The Transgressors
No Land Is Free
Rifle for Rent
Outlaw Town
Mountains Ahead
Rattlesnake Ridge
Sweeney Squadron
On the Old Sante Fe Trail to Siberia
The Bawdy Beautiful
With Women Education Pays off
The English Gardens
The Flooded Dearest
The Beautiful Mankillers of Eromonga
A Matter of Curiosity.
Always Shoot to Kill
The Fifteenth Station
Aid & Comfort to the Enemy
Toughest Lawman in the Old West
Just Any Girl
Mrs. Hacksaw, New Orleans Society Killer
Riding the Dark Train Out
A Cup of the Sun
Seize a Nettle
The Fairbrothers
The Moon & the Thorn
The Crows of Edwina Hill
Land of the Silver Dollar
The Big Family
With Gall & Honey
Mountains of Gilead
The Forger
Take Care of My Roses
A Passion in Rome
The Fleet Rabble
Sam Bentley's Island
The Open Door
A Secret Between Friends
The Huntress
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 92
File
P18
Pl9
P20
P21
P22
P23
P24
P25
P26
P27
P28
P29
R01
R02
R03
R04
R05
R06
R07
R08A
R08B
R09
Author
Anonymous
Anonymous
Spencer Norris
Elizabeth Spencer
Anonymous
Barbara Robinson
Samuel Elkin
William Butler
Ervin D. Krause
Lee McGiffin
Carol Hoover
Robert Carson
Anita Loos
Jean Mercier
Patrick Dennis
Edward Streeter
Evan Esar
James Thurber
John H. Wildman
Leo Lemon
Leo Lemon
S. J. Perelman
Source
No Room in My Heart to For Give
This Cancer Victim May Ruin My Life
Dirty Dog Inn
The White Azalea
A Husband Stealer from Way Back
Something Very Much in Common
The Ball Player
The Pool at Ryusenji
The Snake
Measure of a Man
The Shorts on the Bedroom Floor
My Hero
No Mother to Guide Her
Whatever You Do, Don't Panic
Little Me
The Chairman of the Bored
Humorous English
The Future, If Any, of Comedy
Take It Off
Catch Up With
Something to Talk About
The Rising Gorge
Appendix B: Proportional Placement Statistics
The statistics below come from 40 randomly picked SemCor files. See Cost Function Method 3
in the Cost Function section for more details about these numbers. The POS-POS column represents the
parts of speech of the word pair combination. (NN = Noun, JJ = Adjective, VB = Verb, RB = Adverb)
Table 6: The values for Cost Function Method 3 used in this project
Semantic Relation POS-POS Min
Max
Average
Standard Deviation
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Hypernym
Frequency
0.918299831
0.919316349
0.910315112
0.910315112
0.897634401
0.921952008
0.173931701
0.962140412
0.864021328
0.862272345
0.849278482
0.849278482
0.826850626
0.848030644
0.049567971
0.87887566
0.026870632
0.027024865
0.028957857
0.028957857
0.029391924
0.032030693
0.033316047
0.033752072
NN-JJ
NN-RB
NN-NN
NN-VB
NN-VB
VB-JJ
VB-VB
JJ-RB
0.792254615
0.808804114
0.775888726
0.775888726
0.762541491
0.777054072
0
0.804506863
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 93
Semantic Relation POS-POS Min
Max
Average
Standard Deviation
Frequency
Frequency
Frequency
Coordinate Sister
Frequency
Hypernym
Coordinate Sister
Domain
Synonym
Coordinate Sister
Synonym
Domain
Domain
Coordinate Sister
Coordinate Sister
Antonym
Antonym
Antonym
Antonym
Domain
Domain
Coordinate Sister
Domain
Synonym
Coordinate Sister
Domain
Synonym
Synonym
Synonym
Synonym
Synonym
Antonym
Antonym
Synonym
Antonym
Antonym
Antonym
0.900365277
0.896886676
0.949310958
0.246575342
0.959398173
0.736300313
0.378378378
0.5
0.5
0.738095238
1
1
0.75
1
1
1
1
1
1
1
0.826086957
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
0.839290654
0.827761357
0.879606997
0.110664492
0.867375297
0.517819878
0.185552494
0.029727096
0.183663818
0.422957976
0.01754386
0.047660819
0.05380117
0.046783626
0.062238931
0.035087719
0.035087719
0.035087719
0.035087719
0.035087719
0.148698419
0.125798471
0.104720134
0.109210526
0.244054581
0.105263158
0.228777335
0.122807018
0.545846949
0.131578947
0.582170994
0.140350877
0.152046784
0.216374269
0.207602339
0.469444444
0.368788424
0.035019609
0.036453501
0.040958319
0.042858687
0.062517667
0.06645126
0.083965331
0.105596541
0.107686637
0.11296247
0.131286224
0.156454335
0.159678574
0.170294968
0.183549623
0.184001552
0.184001552
0.184001552
0.184001552
0.184001552
0.185235944
0.211082617
0.252556051
0.264628929
0.272936939
0.277470436
0.285901603
0.295394004
0.311513247
0.331481467
0.334616491
0.3473507
0.353595707
0.384855752
0.390282426
0.423756446
0.439226787
VB-RB
VB-VB
JJ-JJ
VB-VB
RB-RB
NN-NN
NN-VB
NN-VB
VB-VB
NN-NN
VB-RB
NN-JJ
JJ-JJ
NN-RB
VB-RB
NN-VB
VB-JJ
VB-RB
JJ-RB
NN-RB
NN-NN
VB-JJ
RB-RB
RB-RB
NN-JJ
VB-VB
NN-VB
VB-JJ
JJ-JJ
JJ-RB
NN-NN
NN-JJ
RB-RB
NN-JJ
VB-VB
JJ-JJ
NN-NN
0.757327974
0.760132026
0.766452317
0.031847134
0.661223098
0.387156873
0.03125
0
0
0.08
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 94
Appendix C: Sense Distribution Statistics
The statistics below come from the “Brown 1” SemCor files. See Cost Function Method 4 in the
Cost Function section for more details about these numbers. Note that this table assumes the most
common sense is sense 0.
Table 7: The values for Cost Function Method 4 used in this project
Sense Value
Sense Value
Sense Value
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
0.745002967
0.143447519
0.051918954
0.024731353
0.013769042
0.006769587
0.004881351
0.002858328
0.001849772
0.000993421
0.000760418
0.000593476
0.000572185
0.000436927
0.000215302
0.000170978
0.000152483
0.000179044
0.000144862
0.000114338
0.000141419
7.17999E-05
4.29754E-05
7.21787E-05
1.13685E-05
0
3.30413E-05
1.15856E-05
1.10957E-05
1.18544E-05
2.06378E-05
0
0
9.73795E-06
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
Appendix D: Semantic Relation Distribution Statistics
The statistics below come from 40 randomly picked SemCor files. See Cost Function Method 5
in the Cost Function section for more details about these numbers. Note that this table assumes the
most common sense is sense 0.
Table 8: The values for Cost Function Method 5 used in this project
Type
Sense Value
Frequency
Frequency
Frequency
0
1
2
Type
Sense Value
0.781961482 Synonym 0
0.126728106 Synonym 1
0.043575442 Synonym 2
0.708127392
0.14722019
0.058309409
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 95
Type
Sense Value
Type
Sense Value
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
0.020242591
0.011330363
0.005382083
0.003820112
0.002217267
0.001386978
0.000719614
0.000550252
0.000428899
0.000401828
0.000303701
0.000142087
0.000122769
9.93788E-05
0.000126437
9.51612E-05
7.45493E-05
9.84838E-05
4.59405E-05
2.64911E-05
4.80187E-05
6.51078E-06
0
2.37718E-05
6.93931E-06
6.74259E-06
7.47323E-06
1.47131E-05
0
0
5.81201E-06
0
0
0
0
0
0
0
0
0.035265127
0.02488885
0.008951525
0.006850447
0.003343709
0.002257055
0.000917186
0.000415363
0.001272953
0
0.000649926
0.000374687
0
6.48281E-05
0.000302171
0.000120573
0
0.000268828
5.53748E-05
0
7.21131E-05
0
0
0
0
0
0
0.000272292
0
0
0
0
0
0
0
0
0
0
0
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 96
Type
Sense Value
Type
Sense Value
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Frequency
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
42
43
44
45
46
47
48
49
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Synonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
42
43
44
45
46
47
48
49
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
0
0
0
0
0
0
0
0
0.780778412
0.127113255
0.045911925
0.021512378
0.010228877
0.006096897
0.003875595
0.001693082
0.001142567
0.000634825
0.000521345
0.000139175
0.000156363
0.000138005
0
2.79886E-05
9.4065E-06
1.02999E-05
0
0
0
9.6045E-06
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 97
Type
Sense Value
Type
Sense Value
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Hypernym
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Antonym
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0.731642312
0.140189319
0.059664121
0.026321333
0.014048575
0.008563011
0.007495793
0.002041513
0.002195311
0.00124681
0.000714507
0.000830106
0.001219296
0.000939302
0.000651397
0.000227097
0.000161478
0.000455964
0.000287357
0.000232809
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 98
Type
Sense Value
Type
Sense Value
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
Coordinate Sister
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
Domain
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
0.000430405
0
0
0.000109042
0
0
0
0
0.000117354
4.43262E-05
0.000171463
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
Appendix E: SemCor Results
Below are the results for every SemCor file that tags nouns, verbs, adjectives, and adverbs. The
authors ignore all files only tagging verbs. See the results section for more details.
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 99
Table 9: The results of one run of every SemCor file using all the parts of speech
File
Coverage
Fine Recall
Fine Precision Coarse Recall Coarse Precision
Average
br-a01
br-a02
br-a11
br-a12
br-a13
br-a14
br-a15
br-b13
br-b20
br-c01
br-c02
br-c04
br-d01
br-d02
br-d03
br-d04
br-e01
br-e02
br-e04
br-e21
br-e22
br-e23
br-e24
br-e25
br-e26
br-e27
br-e28
br-e29
br-e30
br-e31
br-f03
br-f08
br-f10
br-f13
br-f14
br-f15
br-f16
0.939458723
0.905273438
0.863770977
0.82650143
0.855847688
0.813155386
0.842799189
0.80563654
0.944649446
0.922244094
0.889622642
0.944139194
0.879882813
0.982758621
0.956435644
0.957913998
0.974180735
0.947725073
0.979648474
0.937725632
0.989266547
0.953929539
0.947569113
0.993577982
0.975297347
0.925343811
0.991517436
0.980786825
0.979571984
0.972122302
0.959821429
0.970106075
0.967654987
0.921380633
0.987985213
0.950654582
0.976325758
0.903970452
0.581803497
0.548828125
0.548864758
0.500476644
0.539437897
0.490943756
0.519269777
0.481049563
0.600553506
0.601377953
0.53490566
0.56959707
0.541992188
0.559837728
0.597029703
0.603842635
0.544190665
0.559535334
0.598519889
0.572202166
0.590339893
0.59168925
0.565300286
0.610091743
0.604757548
0.542239686
0.706880302
0.614821592
0.620622568
0.613309353
0.6
0.612343298
0.621743037
0.559923298
0.629390018
0.586102719
0.581439394
0.566020314
0.619021038
0.606256742
0.635428571
0.605536332
0.63029661
0.603751465
0.61612515
0.597104946
0.635742188
0.65208111
0.601272534
0.603297769
0.615982242
0.569659443
0.624223602
0.630372493
0.55861366
0.590398366
0.61095373
0.610202117
0.596745027
0.620265152
0.596579477
0.614035088
0.620075047
0.585987261
0.712927757
0.626865672
0.633565045
0.630897317
0.625116279
0.631212724
0.642525534
0.607700312
0.637043966
0.616525424
0.595538312
0.626149132
0.741617232
0.715820313
0.695952616
0.636796949
0.666364461
0.617731173
0.674442191
0.621963071
0.757380074
0.766732283
0.691509434
0.723443223
0.698242188
0.74137931
0.771287129
0.764867338
0.72591857
0.715392062
0.753931545
0.732851986
0.77549195
0.786238532
0.790856031
0.766634523
0.720038351
0.765409384
0.758325404
0.775047259
0.809297913
0.740186916
0.781683626
0.814629259
0.820781697
0.756397134
0.794545455
0.808173478
0.780943026
0.756141947
0.789075105
0.790722762
0.805714286
0.770472895
0.778601695
0.759671747
0.800240674
0.772014475
0.801757813
0.831376734
0.777306469
0.766246363
0.793562708
0.754385965
0.806418219
0.798471824
0.745158002
0.75485189
0.769593957
0.781520693
0.783905967
0.791320406
0.80734856
0.790258449
0.781477627
0.79313632
0.779843444
0.791505792
0.818618042
0.788059701
0.818005808
0.855789474
0.830279653
0.770594369
0.815298507
0.823979592
0.787128713
0.78028169
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 100
File
Coverage
Fine Recall
Fine Precision Coarse Recall Coarse Precision
br-f17
br-f18
br-f19
br-f20
br-f21
br-f22
br-f23
br-f24
br-f25
br-f33
br-f43
br-f44
br-g01
br-g11
br-g12
br-g14
br-g15
br-g16
br-g17
br-g18
br-g19
br-g20
br-g21
br-g22
br-g23
br-g28
br-g31
br-g39
br-g43
br-g44
br-h01
br-h09
br-h11
br-h12
br-h13
br-h14
br-h15
br-h16
br-h17
0.868937049
0.914201183
0.965041398
0.875121951
0.973256925
0.873441994
0.966730038
0.940186916
0.951871658
0.986486486
0.972407231
0.991877256
0.979206049
0.988614801
0.964717742
0.97790586
0.939252336
0.985808893
0.9453125
0.951661631
0.951866405
0.993269231
0.955555556
0.981768459
0.967615309
0.915151515
0.865927419
0.88334995
0.981354269
0.967615309
0.95559667
0.990224829
0.964749536
0.982010582
0.924599434
0.968430826
0.978947368
0.881709742
0.946666667
0.544891641
0.581854043
0.599816007
0.548292683
0.595988539
0.535953979
0.622623574
0.590654206
0.622994652
0.634169884
0.571836346
0.63267148
0.602079395
0.613851992
0.548387097
0.598463016
0.571962617
0.555345317
0.565429688
0.542799597
0.614931238
0.623076923
0.571428571
0.610756609
0.617271835
0.55959596
0.535282258
0.513459621
0.565260059
0.555446516
0.635522664
0.579667644
0.641001855
0.582010582
0.531573987
0.616527391
0.612280702
0.550695825
0.606153846
0.627078385
0.636461704
0.621544328
0.626532887
0.612365064
0.613611416
0.644051131
0.628230616
0.654494382
0.642857143
0.588062622
0.637852593
0.614864865
0.620921305
0.568443051
0.611984283
0.608955224
0.563339731
0.598140496
0.57037037
0.646026832
0.627299129
0.598006645
0.622098422
0.637931034
0.611479029
0.618160652
0.581264108
0.576
0.574036511
0.665053243
0.585389931
0.664423077
0.592672414
0.574923547
0.63662512
0.625448029
0.624577227
0.640303359
0.810859729
0.782073814
0.767419962
0.760144274
0.81736795
0.777484609
0.760338346
0.826778243
0.773743017
0.811270125
0.743830787
0.74829932
0.756849315
0.810360777
0.768376068
0.754681648
0.78639745
0.791338583
0.759962929
0.735576923
0.653039832
0.763653484
0.682555781
0.730808598
0.795620438
0.83262891
0.716666667
0.72338403
0.706126687
0.707556428
0.719836401
0.696886447
0.734513274
0.721584984
0.747288503
0.714697406
0.744918699
0.731549815
0.709876543
0.816773017
0.80835604
0.778414518
0.799810247
0.852474323
0.7829938
0.815524194
0.832350463
0.802898551
0.819331527
0.757177033
0.77
0.767361111
0.817927171
0.780381944
0.789422135
0.793991416
0.80239521
0.769953052
0.802728227
0.811197917
0.788910506
0.787134503
0.771891892
0.824196597
0.85355286
0.787385554
0.772588832
0.770101925
0.767021277
0.776185226
0.773373984
0.780258519
0.76974416
0.77765237
0.791489362
0.78987069
0.809183673
0.764119601
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 101
File
Coverage
Fine Recall
Fine Precision Coarse Recall Coarse Precision
br-h18
br-h21
br-h24
br-j01
br-j02
br-j03
br-j04
br-j05
br-j06
br-j07
br-j08
br-j09
br-j10
br-j11
br-j12
br-j13
br-j14
br-j15
br-j16
br-j17
br-j18
br-j19
br-j20
br-j22
br-j23
br-j29
br-j30
br-j31
br-j32
br-j33
br-j34
br-j35
br-j37
br-j38
br-j41
br-j42
br-j52
br-j53
br-j54
0.983350676
0.982342007
0.967213115
0.951903808
0.988560534
0.981576254
0.974545455
0.980817348
0.992141454
0.969062784
0.992760181
0.967486819
0.985875706
0.950405771
0.958818263
0.99296394
0.932330827
0.993305439
0.963687151
0.990161002
0.982373678
0.971817298
0.98630137
0.990749306
0.984615385
0.984587489
0.992102665
0.982741117
0.992031873
0.958105647
0.98280543
0.970509383
0.955992509
0.986336465
0.989266547
0.981532779
0.990435707
0.986220472
0.987025023
0.566077003
0.601301115
0.677254098
0.659318637
0.651096282
0.612077789
0.643636364
0.643869892
0.635559921
0.596906278
0.657918552
0.665202109
0.603578154
0.622182146
0.681289167
0.616534741
0.636278195
0.689539749
0.672253259
0.676207513
0.588719154
0.594752187
0.593607306
0.602220167
0.599145299
0.616500453
0.597235933
0.624365482
0.642430279
0.571948998
0.637104072
0.587131367
0.575842697
0.643894108
0.576923077
0.638042475
0.597236982
0.607283465
0.609823911
0.575661376
0.612109745
0.700211864
0.692631579
0.658630665
0.623566215
0.660447761
0.656462585
0.640594059
0.615962441
0.6627165
0.687556767
0.612225406
0.654648956
0.710550887
0.620903454
0.682459677
0.694187026
0.697584541
0.682926829
0.599282297
0.612
0.601851852
0.607843137
0.608506944
0.626151013
0.60199005
0.635330579
0.647590361
0.596958175
0.64825046
0.604972376
0.602350637
0.652813853
0.58318264
0.650047037
0.603004292
0.615768463
0.617840376
0.715820313
0.676441838
0.757236228
0.680705191
0.690140845
0.735439289
0.745694022
0.72037037
0.719626168
0.734032412
0.680467091
0.778474399
0.706766917
0.766144814
0.697368421
0.746421268
0.741573034
0.693501455
0.710659898
0.745711403
0.703858186
0.760692464
0.764981273
0.737588652
0.75123885
0.720772947
0.783619818
0.757904246
0.756911344
0.786825252
0.720039293
0.823751178
0.794144556
0.758992806
0.776785714
0.778077269
0.792051756
0.763343404
0.768939394
0.778958555
0.785471056
0.811
0.793378995
0.773393461
0.801937567
0.80349345
0.77029703
0.769145394
0.776209677
0.758579882
0.791710946
0.775943396
0.803076923
0.770693512
0.776595745
0.784017279
0.78831312
0.787401575
0.810307018
0.75
0.795527157
0.797851563
0.799121844
0.803817603
0.797860963
0.819238901
0.794507576
0.798792757
0.806754221
0.778131635
0.830798479
0.809701493
0.780758557
0.809302326
0.804085422
0.801683817
0.802966102
0.787584869
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 102
File
Coverage
Fine Recall
Fine Precision Coarse Recall Coarse Precision
br-j55
br-j56
br-j57
br-j58
br-j59
br-j60
br-j70
br-k01
br-k02
br-k03
br-k04
br-k05
br-k06
br-k07
br-k08
br-k09
br-k10
br-k11
br-k12
br-k13
br-k14
br-k15
br-k16
br-k17
br-k18
br-k19
br-k20
br-k21
br-k22
br-k23
br-k24
br-k25
br-k26
br-k27
br-k28
br-k29
br-l08
br-l09
br-l10
0.916346154
0.805031447
0.967984934
0.867139959
0.946775844
0.965328467
0.975486052
0.910185185
0.936311787
0.916926272
0.922473013
0.927402863
0.901098901
0.941371681
0.937434828
0.960954447
0.902977906
0.943089431
0.904059041
0.929012346
0.918945313
0.861192571
0.933706816
0.85798237
0.892354125
0.917077986
0.928064843
0.935185185
0.935617861
0.945662536
0.897027601
0.983281087
0.91084855
0.954011742
0.9048583
0.961145194
0.87270974
0.859035005
0.865306122
0.597115385
0.525157233
0.585687382
0.531440162
0.551688843
0.626824818
0.680473373
0.576851852
0.580798479
0.539979232
0.560353288
0.570552147
0.571428571
0.574115044
0.572471324
0.5835141
0.555235351
0.584349593
0.557195572
0.573045267
0.53515625
0.518084066
0.614379085
0.538687561
0.527162978
0.556762093
0.576494428
0.575
0.568016615
0.559580553
0.52866242
0.60815047
0.566058002
0.62035225
0.557692308
0.583844581
0.531340405
0.509933775
0.52244898
0.651626443
0.65234375
0.605058366
0.612865497
0.582702703
0.649338374
0.697573657
0.633774161
0.620304569
0.588901472
0.607446809
0.615214994
0.634146341
0.60987074
0.610678532
0.607223476
0.614893617
0.619612069
0.616326531
0.61683278
0.582359192
0.601589103
0.658
0.627853881
0.590755355
0.607104413
0.621179039
0.614851485
0.607103219
0.591733871
0.589349112
0.618490967
0.621462264
0.65025641
0.616331096
0.607446809
0.608839779
0.593612335
0.603773585
0.694367498
0.666666667
0.711045365
0.690731707
0.756446991
0.667305849
0.771863118
0.745794393
0.792335116
0.787644788
0.796028881
0.731854839
0.766570605
0.775780511
0.741210938
0.729103726
0.777996071
0.783653846
0.754497354
0.790337284
0.793915604
0.714141414
0.672379032
0.657028913
0.782139352
0.760549558
0.77028348
0.796846011
0.741798942
0.708765316
0.770659239
0.776315789
0.686878728
0.762051282
0.780437045
0.787174721
0.793032787
0.803263826
0.801579467
0.768130746
0.767220903
0.777777778
0.789297659
0.777232581
0.763995609
0.798426745
0.793240557
0.832397004
0.798434442
0.802547771
0.75862069
0.78388998
0.786948177
0.784090909
0.766137566
0.817337461
0.788964182
0.789590255
0.805013928
0.820486815
0.780353201
0.776484284
0.743792325
0.797
0.786004057
0.777887463
0.825961538
0.755387931
0.76656473
0.7957814
0.793010753
0.77903044
0.804983749
0.793650794
0.801324503
0.819915254
0.815837937
0.807960199
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 103
File
Coverage
Fine Recall
Fine Precision Coarse Recall Coarse Precision
br-l11
br-l12
br-l13
br-l14
br-l15
br-l16
br-l17
br-l18
br-m01
br-m02
br-n05
br-n09
br-n10
br-n11
br-n12
br-n14
br-n15
br-n16
br-n17
br-n20
br-p01
br-p07
br-p09
br-p10
br-p12
br-p24
br-r04
br-r05
br-r06
br-r07
br-r08
br-r09
0.945863126
0.879728419
0.912641316
0.905365854
0.869
0.891069676
0.873891626
0.914036997
0.902538071
0.920282543
0.938477581
0.903664921
0.869047619
0.923149016
0.835834897
0.895699909
0.907246377
0.892792793
0.919753086
0.971652004
0.956211813
0.878235858
0.951025057
0.921686747
0.925213675
0.82792527
0.919491525
0.958801498
0.922998987
0.934588702
0.903381643
0.956521739
0.576098059
0.537342386
0.535457348
0.545365854
0.508
0.504416094
0.534975369
0.528835691
0.566497462
0.596367306
0.549530761
0.561256545
0.537698413
0.555763824
0.489681051
0.507776761
0.543961353
0.567567568
0.553497942
0.596285435
0.585539715
0.534995206
0.563781321
0.524096386
0.536324786
0.50245821
0.558262712
0.602996255
0.558257345
0.585728444
0.580676329
0.646107179
0.609071274
0.610804851
0.586711712
0.60237069
0.584579977
0.566079295
0.612175874
0.578571429
0.627671541
0.648026316
0.585555556
0.621089224
0.618721461
0.602030457
0.585858586
0.566905005
0.599574015
0.635721493
0.601789709
0.613682093
0.612353568
0.609170306
0.592814371
0.568627451
0.579676674
0.606888361
0.607142857
0.62890625
0.604829857
0.626723224
0.642780749
0.675475687
0.769543147
0.770916335
0.734972678
0.812669683
0.788203753
0.806148591
0.779964222
0.807017544
0.670202507
0.660359508
0.655102041
0.70709147
0.708292683
0.671
0.648675172
0.670935961
0.693144723
0.710994764
0.68452381
0.714151828
0.636022514
0.666971638
0.68115942
0.730630631
0.709876543
0.765395894
0.705656759
0.742596811
0.710843373
0.68482906
0.657817109
0.702330508
0.783057851
0.777108434
0.767110266
0.826887661
0.812154696
0.817316017
0.788426763
0.822201317
0.767955801
0.768722467
0.757075472
0.774774775
0.782327586
0.772151899
0.727973568
0.767756483
0.758333333
0.786790267
0.787671233
0.773604061
0.760942761
0.744637385
0.750798722
0.818365288
0.771812081
0.787726358
0.80349345
0.780838323
0.77124183
0.740184758
0.794536817
0.763824885
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 104
References
ACL-SIGLEX. (2011, March). Retrieved from SENS EVAL: http://www.senseval.org
Basile, P., Degemmis, M., Gentile, A. L., Lops, P., & Semeraro, G. (2007). The JIGSAW Algorithm for Word
Sense Disambiguation and Semantic Indexing of Documents. Proceedings of the 10th Congress
of the Italian Association for Artificial Intelligence on AI*IA 2007: Artificial Intelligence and
Human-Oriented Computing, (pp. 314-325). Rome, Italy.
Billot, M. (2010). Combing Lexical Resources for Text Analysis of Game Walkthroughs. Summer 2010
National Science Foundation REU. Colorado Springs.
Bosch, B. D., Hoste, V., Daelemans, W., & Den, W. V. (2004). GAMBL, Genetic Algorithm Optimization of
Memory-Based WSD. Proceedings of ACL/SIGLEX Senseval-3, (pp. 108-112).
Gerber, M. (2010, October). Retrieved from NLP Software:
http://links.cse.msu.edu:8000/members/matt_gerber/index.php/NLP_software#WordNet_API
Google. (2010). Retrieved from Google Translate: http://translate.google.com/?hl=en&tab=wT#
Hausman, M. (2009). A Dominant Gene Genetic Algorithm for a Substitution Cipher in Cryptography.
CS591 at the University of Colorado. Colorado Springs.
Hausman, M. (2009). A Dominant Gene Genetic Algorithm for a Transposition Cipher in Cryptography.
CS-571 at the University of Colorado. Colorado Springs.
Jiang , J. J., & Conrath, D. W. (1997). Semantic Similarity Based on Corpus Statistics and Lexical
Taxonomy . In International Conference Research on Computational Linguistics , (pp. 19-33).
Leacock, C., & Chodorow, M. (1998). Combining Local Context and WordNet Similarity for Word Sense
Identification. In C. Fellbaum (Ed.), WordNet: An Electronic Lexical Database (pp. 305-332). MIT
Press.
Lesk, M. (1986). Automatic Sense Disambiguation using Machine Readable Dictionaries: How to Tell a
Pine Cone from an Ice Cream Cone. Proceedings of the 5th Annual International Conference on
Systems Documentation, (pp. 24-26). Toronto, Ontario, Canada.
Lin, D. (1998). An Information-Theoretic Definition of Similarity. In Proceedings of the 15th International
Conference on Machine Learning, (pp. 296-304).
Mihalcea, R., & Csomai, A. (2005). SenseLearner: Word Sense Disambiguation for All Words in
Unrestricted Text. Proceedings of the ACL 2005 on Interactive Poster and Demonstration
Sessions, (pp. 53-56). Stroudsburg, PA.
Miller, G. A. (n.d.). WordNet: A Lexical Database for English. Communications of the ACM, 38(11), 39-41.
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 105
Navigli, R. (2007). SemEval-2007 Task 07: Coarse-Grained English All-Words Task. SemEval 2007.
Patwardhan, S. (2006). Using WordNet-Based Context Vectors to Estimate the Semantic Relatedness of
Concepts. Proceedings of the EACL, (pp. 1-8).
Patwardhan, S., Banerjee, S., & Pedersen, T. (2002). An Adapted Lesk Algorithm for Word Sense
Disambiguation Using WordNet. Proceedings of the Third International Conference on
Computational Linguistics and Intelligent Text Processing, (pp. 136-145). London, UK.
Patwardhan, S., Banerjee, S., & Pedersen, T. (2003). Using Measures of Semantic Relatedness for Word
Sense Disambiguation. Proceedings of the Fourth International Conference on Intelligent Text
Processing and Computational Linguistics, (pp. 241-257). Mexico City, Mexico.
Patwardhan, S., Banerjee, S., & Pedersen, T. (2007). UMND1: Unsupervised Word Sense Disambiguation
using Contextual Semantic Relatedness. Proceedings of the 4th International Workshop on
Semantic Evaluations, (pp. 390-393). Prague, Czech Republic.
Patwardhan, S., Banerjee, S., & Pedersen, T. (2007). UMND1: Unsupervised Word Sense Disambiguation
Using Contextual Semantic Relatedness. Proceedings of the 4th International Workshop on
Semantic Evaluations, (pp. 390-393). Prague, Czech Republic.
Princeton University. (2010). About WordNet. Retrieved from About WordNet:
http://wordnet.princeton.edu
Princeton University. (2011, March). Retrieved from The SemCor Corpus:
http://multisemcor.fbk.eu/semcor.php
Rada, R., Mili, H., Bicknell, E., & Blettner, M. (1989, Jan/Feb). Development and Application of a Metric
on Semantic Nets. Systems, Man and Cybernetics, IEEE Transactions, 19(1), 17-30.
Raytheon BBN Technologies, the University of Colorado, the University of Pennsylvania, and the
University of Southern California's Information Sciences Institute . (2011, March). Retrieved
from OntoNotes: http://www.bbn.com/ontonotes/
Resnik, P. (1995). Using Information Content to Evaluate Semantic Similarity in a Taxonomy. Proceedings
of the 14th International Joint Conference on Artificial Intelligence, (pp. 448-453).
Snyder, B., & Palmer, M. (2004). The English all-words task. Proceedings of the 3rd International
Workshop on the Evaluation of Systems for the Semantic Analysis of Text, (pp. 41-43). Barcelona,
Spain.
Tsuruoka, Y., & Tsujii, J. (2005). Bidirectional Inference with the Easiest-First Strategy for Tagging
Sequence Data. HLT/EMNLP, (pp. 467-474).
Wu, Z., & Palmer, M. S. (1994). Verb Semantics and Lexical Selection. Proceedings of the 32th Annual
Meeting on Association for Computational Linguistics, (pp. 133-138). Las Cruces, New Mexico.
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Hausman 106
Zesch, T., & Gurevich, I. (2010). Wisdom of Crowds Versus Wisdom of Linguists: Measuring the Semantic
Relatedness of Words. Natural Language Engineering, 16(01), 1351-3249.
Zhang, C., Zhou, Y., & Martin, T. (2008). Genetic Word Sense Disambiguation Algorithm. Intelligent
Information Technology Applications, (pp. 123-127).
Zhong, Z., Ng, H. T., & Chan, Y. S. (2008). Word sense disambiguation using OntoNotes: an empirical
study. Proceedings of the Conference on Empirical Methods in Natural Language Processing, (pp.
1002--1010). Honolulu, Hawaii.
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation