What Is the “Context” for
Contextual Vocabulary Acquisition?
William J. Rapaport
Department of Computer Science & Engineering
Department of Philosophy
Center for Cognitive Science
NSF ROLE Grant REC-0106338
Outline
• People can figure out a meaning for a word
“from context”
• What does “context” mean in this context?
Definition of “CVA”
“Contextual Vocabulary Acquisition” =def
• the acquisition of word meanings from text
– “incidental”
– “deliberate”
• by reasoning about
– contextual cues
– background knowledge
• Including prior word-meaning hypotheses, language knowledge…
• without external sources of help
– no dictionaries
– no people
CVA: From Algorithm to Curriculum
1. Computational theory of CVA
– Based on:
•
•
algorithms developed by Karen Ehrlich (1995)
verbal protocols (case studies)
– Implemented in a semantic-network-based
knowledge-representation & reasoning system
•
SNePS (Stuart C. Shapiro & colleagues)
2. Educational curriculum to teach CVA
– Based on our algorithms & protocols
– To improve vocabulary & reading comprehension
– Joint work with Michael Kibby
•
Center for Literacy & Reading Instruction
People Do “Incidental” CVA
• We know more words than explicitly taught
– Average high-school grad knows ~45K words
 learned ~2.5K words/year (over 18 yrs.)
– But only taught ~400/school-year
• ~ 4800 in 12 years of school (~ 10% of total)
 Most word meanings learned from context
– “incidentally” (unconsciously)
• How?
People Also Do “Deliberate” CVA
•
•
•
•
•
•
•
You’re reading;
You understand everything you read, until…
You come across a new word
Not in dictionary
No one to ask
So, you try to “figure out” its meaning from “context”
How?
– guess? derive? infer? deduce? educe? construct? predict? …
– our answer: Compute it! Via inferential search of “context”/KB
• But what KB?
CVA as Cognitive Science
• Studied in:
–
–
–
–
–
AI / computational linguistics
Psychology
Child-language development (L1 acquisition)
L2 acquisition (e.g., ESL)
Reading education (vocabulary development)
• Thus far: “multi-”disciplinary
• Not yet: “inter-”disciplinary!
What does ‘brachet’ mean?
(From Malory’s 15th century Morte d’Arthur [page # in brackets])
1. There came a white hart running into the hall with a white
brachet next to him, and thirty couples of black hounds
came running after them. [66]
•
•
People:
Computer:
•
brachet = animal? inanimate object? don’t know.
brachet = physical object
(because only physical objects have color)
2. As the hart went by the sideboard, the white brachet bit
him. [66]
•
•
People:
Computer:
•
brachet = animal
brachet = animal
(because only animals bite)
Malory, continued
3. The knight arose, took up the brachet and rode
away with the brachet. [66]
•
•
People:
Computer:
•
brachet = animal / small animal
brachet = small animal
(because: picked up and carried)
4. A lady came in and cried aloud to King Arthur,
“Sire, the brachet is mine”. [66]
•
•
People:
Computer:
•
brachet = pet / small, valuable animal
brachet = small, valuable animal
(because: what’s wanted is valuable)
Malory, continued
10. There was the white brachet which bayed at him fast. [72]
•
•
People:
Computer:
•
brachet = dog
brachet = hound (i.e., dog that hunts)
(because only hounds, which are hunting dogs, bay)
18. The hart lay dead; a brachet was biting on his throat, and
other hounds came behind. [86]
•
•
People:
Computer:
•
brachet = hound
brachet = hound (i.e., dog that hunts)
(because “x and other y”  x is a y)
How (Not) to Teach CVA:
Vague Strategies
• Clarke & Nation 1980: a “strategy” (algorithm)
1.Look at word & context; determine POS
2.Look at grammatical context
• E.g., “who does what to whom”?
3.Look at wider context
• [E.g., for clues re: causal, temporal, class-membership, etc.]
4.Guess the word; check your guess
Vague strategies:
“guess the word”
=
“then a miracle occurs”
•
•
Surely,
we computer scientists
can “be more explicit”!
A More Precise, Teachable Algorithm
• Treat “guess” as a procedure call
– Fill in the details with our algorithm
• Convert the algorithm into a curriculum
– To enhance students’ abilities to use deliberate
CVA strategies
Figure out meaning of word from what?
• context (i.e., the text)?
– Werner & Kaplan 52, McKeown 85, Schatz & Baldwin 86
• context and reader’s background knowledge?
– Granger 77, Sternberg 83, Hastings 94
• context including background knowledge?
– Nation & Coady 88, Graesser & Bower 90
• Note:
– “context” = text  context is external to reader’s mind
• Could also be spoken/visual/situative (still external)
– “background knowledge”: internal to reader’s mind
• What is (or should be) the “context” for CVA?
Some Proposed Preliminary Definitions
(to extract order out of confusion)
• Unknown word for a reader =def
– Word or phrase that reader has never seen before
– Or only has vague idea of its meaning
• Different levels of knowing meaning of word
– Notation: “X”
Proposed preliminary definitions
• Text =def
– (written) passage
– containing X
– single phrase or sentence … several paragraphs
Proposed preliminary definitions
• Co-text of X in some text =def
– The entire text “minus” X; i.e., entire text surrounding X
– E.g., if X = ‘brachet’, and text =
• “There came a white hart running into the hall with a white
brachet next to him, and thirty couples of black hounds came
running after them.”
Then X’s co-text in this text =
• “There came a white hart running into the hall with a white
______ next to him, and thirty couples of black hounds came
running after them.”
– Cf. “cloze” tests in psychology
• But, in CVA, reader seeks meaning or definition
– NOT a missing word or synonym: There’s no “correct” answer!
– “Co-text” is what many mean by “context”
• BUT: they shouldn’t!
Proposed preliminary definitions
• The reader’s prior knowledge =def
– the knowledge that the reader has when s/he
begins to read the text
– and is able to recall as needed while reading
• “knight picks up & carries brachet” ? small
• Warnings:
– “knowledge”  truth
• so, “prior beliefs” is better
– “prior” vs. “background” vs. “world”, etc.
• See next slide!
Proposed preliminary definitions
• Possible synonyms for “prior knowledge”,
each with different connotation:
– Background knowledge:
• Can use for information that author assumes reader to have
– World knowledge:
• General factual knowledge about things other than the text’s topic
– Domain knowledge:
• Specialized, subject-specific knowledge about the text’s topic
– Commonsense knowledge:
• Knowledge “everyone” has
– E.g., CYC, “cultural literacy” (Hirsch)
• These overlap:
– PK should include some CSK, might include some DK
– BK might include much DK
Steps towards a
Proper Definition of “Context”
•
Step 1:
–
The context of X for a reader =def
1. The co-text of X
2.
•
“+” the reader’s prior knowledge
Both are needed!
–
After reading:
•
“the white brachet bit the hart in the buttock”
most subjects infer that brachets are (probably) animals, from:
•
•
–
That text, plus:
Available PK premise: “If x bites y, then x is (probably) an animal.
Inference is not an enthymeme! (because …)
Proper definition of “context”:
• But (inference not an enthymeme because):
– When you read, you “internalize” the text
• You “bring it into” your mind
• Gärdenfors 1997, 1999; Jackendoff 2002
– This “internalized text” is more important than the
actual words on paper:
• Text:
• Misread as:
“I’m going to put the cat out”
“I’m going to put the car out”
– leads to different understanding of “the text”
– What matters is what the reader thinks the text is,
• Not what the text actually is
• Therefore …
Proper definition of “context”:
• Step 2:
– The context of X for a reader =def
• A single KB, consisting of:
1. The reader’s internalized co-text of X
2.
“+”
the reader’s prior knowledge
Proper definition of “context”:
• But: What is “+”?
– Not: mere conjunction or union!
– Active readers make inferences while reading.
• From text = “a white brachet”
& prior commonsense knowledge = “only physical objects have color”,
reader might infer that brachets are physical objects
• From “The knight took up the brachet and rode away with the brachet.”
& prior commonsense knowledge about size,
reader might infer that brachet is small enough to be carried
– Whole > Σ parts:
• inference from [internalized text + PK]  new info not in text or in PK
• I.e., you can learn from reading!
Proper definition of “context”:
• But: Whole < Σ parts!
– Reader can learn that some prior beliefs were mistaken
• Or: reader can decide that text is mistaken (less likely)
• Reading & CVA need belief revision!
• operation “+”:
– input: PK & internalized co-text
– output: “belief-revised integration” of input, via:
• Expansion:
– addition of new beliefs from ICT into PK, plus new inferences
• Revision:
– retraction of inconsistent prior beliefs together with inferences from them
• Consolidation:
– eliminate further inconsistencies
Prior Knowledge
PK1
PK2
PK3
PK4
Text
Prior Knowledge
PK1
PK2
PK3
PK4
Text
T1
Integrated KB
internalization
PK1
I(T1)
PK2
PK3
PK4
Text
T1
B-R Integrated KB
internalization
PK1
I(T1)
PK2
inference
PK3
PK4
Text
P5
T1
B-R Integrated KB
internalization
PK1
I(T1)
PK2
inference
PK3
Text
P5
PK4
I(T2)
P6
T1
T2
B-R Integrated KB
Text
internalization
PK1
I(T1)
PK2
T1
T2
inference
PK3
P5
PK4
I(T2)
P6
I(T3)
T3
B-R Integrated KB
Text
internalization
PK1
I(T1)
PK2
T1
T2
inference
PK3
P5
PK4
I(T2)
P6
I(T3)
T3
Note: All “contextual” reasoning is done in this “context”:
B-R Integrated KB
internalization
PK1
P7
Text
I(T1)
PK2
T1
T2
inference
PK3
P5
PK4
I(T2)
P6
I(T3)
T3
Proper definition of “context”:
• One more detail: X needs to be internalized
• Context is a 3-place relation among:
– Reader, word, and text
• Final(?) def.:
– Let T be a text
– Let R be a reader of T
– Let X be a word in T (that is unknown to R)
– Let T-X be X’s co-text in T.
– Then:
• The context that R should use to hypothesize a meaning
for R’s internalization of X as it occurs in T =def
– The belief-revised integration of R’s prior knowledge
with R’s internalization of T-X.
This definition agrees with…
• Cognitive-science & reading-theoretic views of
text understanding
– Schank 1982, Rumelhart 1985, etc.
• & KRR techniques for text understanding:
– Reader’s mind modeled by KB of prior knowledge
• Expressed in KR language (for us: SNePS)
– Computational cognitive agent reads the text,
• “integrating” text info into its KB, and
• making inferences & performing belief revision along the way
– When asked to define a word,
• Agent deductively searches this single, integrated KB for
information to fill slots of a definition frame
– Agent’s “context” for CVA = this single, integrated KB
Distinguishing Prior Knowledge from Integrated Co-Text
• So KB can be “disentangled” as needed for
belief revision or to control inference:
• Each proposition in the single, integrated
KB is marked with its “source”:
– Originally from PK
– Originally from text
– Inferred
• Sources of premises
Some Open Questions
• Roles of spoken/visual/situative contexts
• Relation of CVA “context” to formal
theories of context (e.g., McCarthy, Guha…)
• Relation of I(T) to prior-KB; e.g.:
– Is I(Ti) true in prior-KB?
• It is “accepted pro tem”.
– Is I(T) a “subcontext” of pKB or B-R KB?
• How to “activate” relevant prior knowledge.
• Etc.
Summary
• People can figure out a meaning for a word “from context”,
where…
• “Context” = belief-revised integration of:
– reader’s prior knowledge, with
– internalized information from the text
• This clearer concept of relevant notion of “context” will
help us:
– evaluate other research
– develop our curriculum