Is Question Answering an
Acquired Skill?
Ramakrishnan, Chakrabarti, Paranjpe,
Bhattacharyya
Paper presentation: Vinay Goel
Introduction
• Question Answering (QA) system
• Most QA systems are substantial team efforts
– Difficult to reproduce a well tuned QA system from
scratch, gauge the benefit of new algorithmic
ideas, new corpora and new languages
• QA systems
– Complex building blocks like taggers and parsers
– Lashed together with customized “glue”
– Crucial knobs best preset by QA specialists
Goal
• Decompose the QA task cleanly into
discovering features and learning to score
answer snippets
• QA system
– Performs fast, shallow processing of corpus
– Structures the scoring task using features and
learners
– Trains its scoring algorithm from a past history of
questions and vetted answers
– Can include side information (Wordnet etc.)
– Reuses expertise accumulated from one corpus to
a new corpus
Noisy simulation perspective
• In a structured database with a suitable schema and
a structured query language, information needs can
be expressed clearly
• See QA as a transformation of this process by adding
natural language from an unknown generative
process, for both query and data
• Given a question, discover structured fragments in it
– Extract selectors which will appear almost unchanged in an
answer passage
– Extract atype clues, which tell what else to look for in a
passage that satisfied all selectors
Typical connections between
a question and answer
Atype
• Minimal subclass of entities which will
answer a question
• Two representations important to factoid
QA
– Atype as synset
– Atype as surface patterns
Atype as synset
• Q: Name an animal that sleeps upright
• A: horse
• Wordnet helps recognize that horse is
an instance of animal
• Most answers which are common nouns
are assisted by this representation
Atype as surface patterns
• Infinite or very large domains such as
numbers, person names, place names
etc. cannot be covered by Wordnet
• Logically augment Wordnet to add
connections from synsets to pattern
matchers such as “at DD:DD” or “Xx+
said” etc.
From the question to an atype
• Set of common “wh-words”
• Questions starting with when, where
and who immediately reveal their
expected atypes
• Word after how is almost always a clue
• Questions using using what and which
mention atype directly
Shallow parsing to extract
atype
• Shallow parsing involves finding noun
phrases, modifiers and attachments between
phrases
• Purely based on POS tags
• Strategy for locating atype clues from “what”
and “which” questions:
– Head of NP appearing before the auxiliary / main
verb if it is not a wh-word
– Otherwise, head of NP appearing after the verb
Learning to map atype
• When, where, who and how do not
directly use a term that describes a
synset
• Augmented synsets based on surface
patterns (DDDD) may come handy
• Devised a learning module to help
compile mappings between short token
sequences in questions to atypes
Selectors
• Second blank in the SQL query
select…where…
• In QA, simply a set of words in the
question that are expected to appear
unchanged in the answer passage
Identifying selectors
• Choice of features
– POS
– POS assigned to left and right neighbors
– Whether the word starts with an uppercase letter
–
–
–
–
Whether the word is a stopword
Some version of IDF
How many senses the word has in isolation
For a given sense, how many other words
describe the sense
How to use selectors
• Two places
– Pad the initial keyword query
– Rerank the candidate phases
• Experiments insist that the response by
the word search engine (Lucene)
– Contains all selectors
– Use OR over other question words
Learning to score passages
• If (q,r) is a positive instance, it is
expected that
– All selectors match between q and r
– r has an answer zone a which does not
contain selectors
– The linear distance between a and
matched selectors in r, tend to be small
– a has strong Wordnet-based similarity with
the atype of q
Overall architecture
Experiments
• TREC QA track
• Picked sliding windows of three
sentences as passages
• Questions and passages were
tokenized using GATE
• For learning tasks, used J48 decision
tree and the logistic regression
packages in WEKA
Extracting atypes from shallow
parses
Spotting selectors in questions
Passage Reranking
performance
MRR improvement via
reranking
Training on corpus of another
year
Conclusion
• QA system built by wrapping logic
around text indexers, taggers, shallow
parsers and classifiers
• Simple assembly of building blocks
• Future work involves improving
performance of different blocks