New Approaches to Story
Modeling for Understanding,
Generation and Summarization
David K. Elson
December 6, 2004
What kind of a talk is this?
• About modeling
– In principle
• About stories
– Theory and practice
• About stories and NLP
– History
– New news summarizer
• Initial results
• Not about generation background
2
The Plot
1.
2.
3.
4.
5.
6.
7.
A Word on Models
Narrative for News?
Related Work
The Big Idea
Waxing Philosophic
Back to News
Wrapping Up
3
The Plot
1. A Word on Models
2.
3.
4.
5.
6.
7.
Narrative for News?
Related Work
The Big Idea
Waxing Philosophic
Back to News
Wrapping Up
4
What is a model?
• A discrete set of conceptual categories meant
to quantize the range of expected values of a
natural phenomenon.
• A machine or human can apply the model to
an instance of the phenomenon by assigning
a category to each token to a category
(“parsing”).
– An adaptive system updates its model to fit new
patterns in new data.
5
Example of a model
Phenomenon
Model
ABCDEFGHIJKLMNOPQRSTUVWXYZ
Phenomenon,
parsed
according to
model
DATE DID SOMETIMES WITHOUT
PROBABLY AFTER PROBABLY SOON
AMONG ITSELF ONCE SENT OUT
HOUSE GOING SAD UNDER UPON
6
More on models
• We as humans work by giving inductive
conclusions to the categories in our models
based on statistical patterns from experience.
– What is a tree? What is blue?
• For example, we have seen enough houses
with bedrooms to expect a bedroom when
we discuss a house.
– This is an expectation model.
7
How many distinct categories?
• Too few, and our deductions are wrong.
– “Does the House of Representatives have a
bedroom?
• Too many, and we can’t make inductions.
– “Is there going to be a room for me to sleep in at
Grandma’s house?”
8
Equivalence in NL modeling
• Statistical language modeling
– Unigrams D(w1)
• Enough data to see patterns, but
• So coarse that the data are heterogeneous,
and the inductions from patterns are wrong.
– Trigrams D(w3|w1,w2)
• Specific enough to for patterns to be useful
(high information content), but
• Too many categories, too little data to see
patterns.
9
More on expectation models
• What am I saying here?
• Parse using Model 1 (English, millions of categories)
– Versatile, but not very useful
• Parse using Model 2 (fifty categories)
– Model doesn’t fit data: neither versatile nor useful
• Parse using Model 3 (three categories)
– Useful, but not very versatile
10
Bigger problem
• Humans are good at creating and maintaining
useful models subconsciously to make
knowledge.
• They then try to pass on that knowledge to
the computers, by “arbitrarily” defining
categories.
– Language is messy.
– Every model “compiles,” whether or not it’s right.
11
Modeling nature with machine learning
It’s C2!
ML
Features F1,
F2 … F2000
Model: this is C1, C2, or C3
12
We tend to throw ML at a problem.
I think it’s C1…
ML
Features F1,
F2 … F2000
13
Is This A Bad Idea?
• If we were classifying rocks, then no.
– Phenomenon is static and well-understood.
– Emphasis should be on new feature detection.
• But we are modeling human language.
– Incredibly complex
• emotion, culture, upbringing, a wide range of intentions.
• We are more comfortable and skilled with
computers than with philosophy.
14
How would you model these
NLP problems?
•
•
•
•
•
•
Fact or opinion?
Relevant sentence or irrelevant?
News or editorial?
How to generate text with style?
How to detect tone?
How to detect emotion?
15
The good news
• There are many people in other fields whose
whole problem is your first step: finding a
model.
– Theories in journalism, linguistics, psychology
• You can sometimes even ask questions to the
people who created your corpus!
– “Poems are made by fools like me, but only God
can make a tree.”
16
The main point
• Look closely at your data.
• Pick practical, yet useful distinctions for your
categories that fit the data.
– Your final research goal, without grey areas, may
be an impractical model to train on.
• Look at the non-technical literature.
• But don’t fall into a rabbit hole of modeling!
– Remember, the model problem will never be
solved!
17
The Plot
1. A Word on Models
2. Narrative for News?
3.
4.
5.
6.
7.
Related Work
The Big Idea
Waxing Philosophic
Back to News
Wrapping Up
18
My research
• A model for stories and storytelling.
• Abstract, but potentially applicable to many
NL fields.
• I’m also applying it now to news
summarization.
19
Untapped assumptions for news?
Text
Narratives
Narrative News
Articles
News
20
Story Form for News
Summarization?
• Many news articles are narrative in form
– Structural restrictions
• Frequently use quoted opinions, statements of history,
character descriptions…
– Content restrictions
• Every news article happens because an editor thought
the story would be interesting. How did they decide?
• If stories can be modeled, so can news
– News summaries are minimal retellings of the news story
– If we can tell a story, we can write a summary
21
News is Newsworthy When
It’s a Good Story
JERUSALEM (CNN) -- The Israeli military has embarked on a
wide-ranging raid in Jenin after getting tips that the Palestinian
militant group Islamic Jihad was planning to carry out terror
attacks against Israel from the northern part of the West Bank,
the Israel Defense Forces said Thursday.
• Syntactic: description of causally related events
– Consequence followed by cause
• Semantic: Narrative characters and goals
– “Avoidance of terror” goal, “preemptive strike” plot
22
The big questions
• What is a story?
• Is there a model for stories that we use as humans
and journalists?
– Is there a small set of possible stories?
– Do we have a story expectation model that fills in the gaps?
• Can the model be encoded?
– Can a computer understand a story or tell a story?
– Can it do it well enough to be useful for news summarization
or story generation?
23
The Plot
1. A Word on Models
2. Narrative for News?
3. Related Work
4.
5.
6.
7.
The Big Idea
Waxing Philosophic
Back to News
Wrapping Up
24
Everyone wants a model
Story
Generation
Story
Understandin
g
Creative Hack
Theory
Psychology
Model of
Stories
Literary
Theory
Structuralist,
Constructivist and
Semiotic Theory
25
Many Models, Many Dimensions
It’s a GRAMMAR!
It’s about FRAMES!
SEMIOTICS! PLOT FRAGMENTS!
It’s about FABULA!
CHARACTER GOALS!
26
An essential distinction to start with
Text structure:
Hook, elaboration, background (news)
Rising/falling action, climax, coda (fiction)
Discourse
Narrative semantics:
Heroes, villians, goals, obstacles, plot
arcs, themes, morals
Story
Raw timeline:
Actions, events, characters
Fabula
Observed
Implied
Most models are models of one of the
top two layers. Some conflate them all.
27
For example
Discourse
Story
Fabula
28
Fabula
John and Joe’s Story: Fabula
0
•
•
•
FABULA
•
•
TIME
•
LINE
•
•
•
•
John and Joe’s wife, Sue, have an affair.
John moves to Vermont.
John gets a job in broadcasting.
Joe finds out about the affair and divorces Sue.
John becomes a well known anchor.
Joe becomes depressed.
John and Sue get married and have kids.
John is stalked by a mysterious person.
John hires a PI and a bodyguard.
Joe is arrested for stalking John.
29
Layers of narrative
Discourse
Story
Fabula
30
Story
• The fabula is perceived by a focalizer
– Perceived by either (point of view)
• An omniscient being
• A character in the fabula
• Some nested combination (e.g., Sherlock Holmes)
– Events can be dismissed or emphasized
– Events may be perceived out of order
31
Story
• Narrative semantics are applied a fabula to
make it palatable and interesting
– Invoke our own models of what a story is
• Based on our life exposure to stories
• We need someone to “root for,” a conflict to care about
– My research: What is that model?
• What are the classes of plots, characters, themes that we
expect?
• Natural to writers, known to all readers, but hard to
identify
32
Story
John and Joe’s Story: Focalization #1
“Famous Newcaster Threatened by Stalker”
– John is a funny, easygoing guy and a popular
newscaster with a loving wife and kids.
– One day John is heckled while on air; later he is
followed home. The perpetrator escapes.
– John’s pet cat is abducted. He feels very
frightened.
– John hires a PI and a bodyguard. This creates
conflict with his family.
33
Story
John and Joe’s Story: Focalization #2
“Betrayed Friend Seeks Revenge Against
Arrogant Famous Newscaster”
– A disheveled drifter, Joe, arrives in Vermont.
– Joe describes his mission to a waitress:
– John stole his wife from him.
– John unapologetically left and got famous.
– Joe demands compensation from John.
34
A brief sidestep: what narrative
semantics did we just discover?
•
•
•
•
•
•
•
•
The personal revenge plot
Love as a device for battle
The sympathetic, comfortable hero
The sympathetic, downtrodden hero
The arrogant, selfish, rich villain
The deranged, threatening villain
The stalking and theft of property plots
The pet cat as beloved property
Is there a complete list?
How is it organized?
35
Layers of narrative
Discourse
Story
Fabula
36
Discourse
• The surface depiction of the story
• Many “genres” of narrative discourse:
–
–
–
–
–
–
–
Written prose
Stage drama
Cinema
Poetry
Hieroglyphs
Shadow puppets
News article
• Each has its own structural characteristics
37
Discourse
The narrative agent’s tools in prose
•
•
•
•
•
•
•
•
•
Descriptions of actions
Quotations of speech
Selection of details
Comments and descriptions
Imagery
Foreshadowing
Grammar and word usage
Ideological pontifications
Historical digressions
38
Many Models, Many Dimensions
It’s a GRAMMAR!
It’s about FRAMES!
SEMIOTICS! PLOT FRAGMENTS!
It’s about FABULA!
CHARACTER GOALS!
39
Literary
Theory
Campbell
Plot Fragments
• 1949: Mythologist Joseph Campbell publishes Hero
with a Thousand Faces
– Societal and psychological importance of myth
– Myth as cultural heritage
– Myth archplots:
• Call to adventure and refusal of call
• Threshold crossing, dragon-battle, dismemberment,
crucifixion, abduction, wonder journey, whale’s belly
• Return, rescue, resurrection, threshold struggle
• Sacred marriage, father atonement, apotheosis
• Still not formal or complete enough to be useful
40
Literary
Theory
Propp
Plot Fragments
• 1966: Russian Formalist Vladimir Propp looks at the
Morphology of a Folk Tale
• Most Russian folk tales draw from a pool of 31 plot
fragments
–
–
–
–
–
–
The hero leaves home
The villain attempts to deceive the victim
The hero and villain join in direct combat
The false hero or villain is exposed
The villain is exposed
The hero marries and ascends to the throne
• A grammar dictates how the fragments fit together
41
Story
Understanding
Early
Software
Plot Fragments
• SAM: Script Applier Mechanism (Cullingford, 1978)
– Reads NL text into causal chains
– Rigid knowledge base and inference engine
– Small input tolerance (only handles trivial stories)
• PAM: Plan Applier Mechanism (Wilensky 1978)
– Models around character goals, infers motivations
– Still heavy reliance on inference engine, knowledge base
– Still can’t handle real stories
• TALE-SPIN: first generator (Meehan 1979)
– On-the-fly problem solving
– Stories are often not interesting, little conflict
42
Story
Understanding
•
Mid-80s
Marker Passing
FAUSTUS (Norvig 1987) and ATLAST (Eiselt 1987)
are marker passing systems
–
–
–
Less bound by rigid scripts and frames than SAM and
PAM
Markers are “passed” through the semantic network until
they run out of “energy”
Strong inferences are drawn, weak ones dismissed
43
Story
Understanding
•
•
Rumelhart
Grammars
1975: David E. Rumelhart
Syntactic rules for story structure:
– Rule 1: Story = Setting + Episode
– Rule 6: Internal Response = (Emotion | Desire)
– Rule 10: Preaction = Subgoal + (Attempt)*
•
Summarization rewrite rules
–
•
Summary(CAUSE[X,Y]) = “Instrument(X) caused (Y)”
Grammar is not powerful, but very influential
–
–
Is a grammar the right approach?
Large debate (e.g., Wilensky, Black)
44
Story
Generation
MINSTREL
Frames
• Scott Turner 1992
• Story generation and surface rendering
• Models the human process of storytelling
– Consistency, suspense, fulfillment of themes
• Appears “creative” with case-base reasoning
– Recalls, adapts, and applies previous solutions
• Domain: King Arthur’s Court
– Goals include obtaining wives, slaying dragons
45
“Today”: Rebirth
• Away from the symbolic, toward the practical
• 1999: AAAI Symposium on Narrative Intelligence
–
–
–
–
–
MIT: Narrative as retold in social networks (Usenet)
MIT Media Lab: Converting travelogues to narrative form
Swedish Institute of CS: Browsing the Web in narrative
Brenda Laurel: interfaces with characters, metaphors
IIT (Canada): Narrative with autonomous agents
– New definitions, new models
• Based on conflict between characters (Szilas)
• Based on the reader’s reaction (Bailey)
• Based on new grammars (Lang)
46
Story
Generation
BRUTUS.1
Grammars
• 2000: Selmer Bringsjord and David Ferrucci (RPI)
• Story generation and discourse rendering
• Formal representation of dramatic themes
– Betrayal, self-deception
• Renders with both grammars and simulations
– Literary techniques (irony, metaphor)
• Domain: a student’s dissertation defense
– Very small!
– Don’t know how “serious” this is, but a lot of press!
• Optional New York Times reading
47
My Take On This History
• Use of narrative semantics conflated with knowledgebased AI for news
– Taboo to even cite!
• I believe we can separate them
– Practical partial understanding with statistically trained story
models.
• Plot-fragments: too few categories to be versatile
– No internal consistency
• Debate over story grammars saw them as purely
linguistic (surface discourse)
– But texts and fabulas can be modeled independently.
48
The Plot
1. A Word on Models
2. Narrative for News?
3. Related Work
4. The Big Idea
5. Waxing Philosophic
6. Back to News
7. Wrapping Up
49
All The World’s A Stage
• Let’s step away from the computer for a while.
– Goal is model of storytelling. Tool is computer AI.
• Let’s focus on the middle layer (“Story”) and model
the ways we can perceive and portray events in the
fabula.
• An interesting story has:
– Characters with whom we identify;
– Goals for the characters that we want them to obtain;
– Difficulties that the characters confront in pursuing their
goals.
50
Let’s just list the wants!
•
•
•
•
•
•
•
•
•
•
•
Life
Physical health
Mental health
To be physically home
Provision of a want for family
The wants of clan (e.g., ethnic)
Freedom of movement
Freedom of behavior (e.g., political freedom)
Companionship
Privacy
Etc.
51
A Rhetorical Grammar
•
•
•
•
•
STORY: (STATE, ACTION)+, WORLD
STATE: (CHARACTER, GOAL+, ENVIRONMENT)+
CHARACTER: (CHARACTER+ or )
GOAL: WANT+, subgoal(GOAL)
WANT: (life) or (physical health) or (provision(WANT,
CLAN)) or (protection(WANT)) or (physical journey)
or (privacy) or (acquisition(symbol(WANT)))…
• CLAN: family, ethnic group, gang
52
A Rhetorical Grammar
• The abstract plot of Finding Nemo with the
particulars (names, places) as terminals in
the parse tree:
– State: [Marlin] wants [provision (health (Nemo))]
with SUBGOAL of [physical journey (ocean)]
– [Nemo] wants [to be physically home] and
[companionship] and [freedom of behavior from
[Marlin]].
53
The horoscope effect
• Like seeing your life in a horoscope, the model is not
“proven” because some story fits.
• This is where the number of categories point matters
– Like any model, tradeoff between versatility and specificity
• Impossible to document the model that can write the
full script of Finding Nemo
– Too much real world knowledge needed
• But we can make this model progressively more
detailed until we hit the knowledge barrier
– The building blocks of stories that we (or a computer) can
combine in novel ways that still work
54
Value Systems
• Each character assigns a comparable value to all her
wants.
• This captures many situations:
– Plots: Sacrifice of one want for another
• Suicide bombers, whistle-blowers
– Characters: Unusual or “evil” value systems
• We are surprised at a choice of sacrifice
– Stories of total commitment to a hobby, isolated living, etc
– Ruthless villains value others’ lives lower than their wants
• A character changes from a “bad” to “good” system
– Comes to prioritize family over career
55
Clans
• Any subset of the population with a common bond
forms a clan.
– Ethnic group, socio-economic group, special interest group,
family, gang, gender…
• This captures many situations:
– Plots: love across clan boundaries, struggling to join or leave
a clan, war between clans, protecting or saving a clan
member from danger
– Characters: conflicted on a clan boundary, the leader of a
clan, going against the clan’s traditions
– Environment: the clan in mortal danger
56
Categories of Human Flaws
• You internalize your emotions and don't let anyone help you.
• You judge and/or stereotype others too quickly based on too
little information.
• You hold grudges long beyond their useful lifetime.
• You fear what you don't know or understand.
• You defend other people's ideas at the expense of your own.
• You don't know your self-worth.
• You get upset with yourself too often and for too trivial matters.
• You are incapable of asserting and defending yourself.
• You believe you have no flaws.
• You believe it is your duty to help others overcome their faults.
• Etc.
57
Categories of Personality Contrasts
•
•
•
•
•
•
•
•
•
•
•
•
Organized, logical introvert vs. free-associating extrovert
Private and closed off vs. connected and generous
Type-A (driven) vs. type-B (patient)
Highly educated and analytical vs. ignorant and impulsive
Looking toward the future vs. longing for the past
Cynicism of age vs. romanticism of youth
Risk taker vs. play-it-safer
Popular (or famous) vs. unpopular
Talented vs. hack
Talented vs. gifted
Pragmatist vs. true believer
Etc.
58
We already use story models
• We are documenting a mental expectation model
• How would you close these stories?
– The beautiful princess, locked away in her tower so that she
would never know love, called out for the handsome prince.
– Harry looked for the keys under the car.
• On art vs. entertainment…
59
Hypothetical stack trace
• [Harry] does PLOT[ [physical search] for
[symbol[ability[freedom of movement]]] ]
– Why?
– Plot common for WANT[freedom of movement]
– [freedom of movement] a common REQUIREMENT for WANT[be
at point X]
– Therefore, this is subgoal[WANT] where WANT requires Joe at
point X
• If this is the human trace, can the computer mimic?
– Narrative inference, statistically trained
– Too much inference needed to parse from text directly.
• Car keys or house keys? Car keys are always a symbol?
60
Differences from prior work?
• Isolation to only middle (“Story”) layer
– More versatile, complete, powerful than plot fragment
models like Campbell
• Underlies stories in all genres
– No conflation of story and linguistic telling of story
– Smaller, more abstract and portable model than hard AI
– Not practically implementable as a free-text parser
• “If person A takes person B’s car with consent, it is a
friendly favor; otherwise, it is a theft that illustrates these
flaws and gives B a reclamation of property want…”
– Not knowledgeable enough for generation
61
So what did we accomplish?
• Much work in creating abstract model
– As written notes
• Model itself may be a research contribution
– Formal language of storytelling
– Not sure of what community, cognitive science?
• Computational story-logic engine in progress
– Might apply and specialize to various NLP problems
• News
• Question-answering
• Other specialized generation
62
The Plot
1.
2.
3.
4.
A Word on Models
Narrative for News?
Related Work
The Big Idea
5. Waxing Philosophic
6. Back to News
7. Wrapping Up
63
On creativity
• Creative writers hate these projects.
–
–
–
–
“You can’t put creativity in a box!”
“How can one list serve every kind of story?”
“You’re just finding cliches!”
“Making computers creative will destroy a part of humanity!”
• Think of this as a dictionary and grammar of stories.
– Every English word has been discovered and defined, but we
can still write novel sentences.
– Billions of stories possible with model; only a few work.
– Creativity lies in selection, application to a setting and
execution (telling the story well)
– This project has vastly improved my own creative writing.
• Building with raw wood and concrete instead of preset rooms.
64
On creativity
• Every story is formulaic, for sufficiently small
formulas.
– The best writers read, experience the most.
• A a mental sense plots, characters, dynamics that work.
• They use this model when they write.
– To document this model does not destroy it.
• Any more than documenting English language does.
• The model is flexible enough to create bad stories, too.
– Computers will never be able to tell stories as
well as humans.
65
The almighty story
• Narrative semantics used everywhere!
– Journalists’ creed: “tell a good story”
• Convey complex issues with personal stories
– Nonfiction authors: “find the story” in the subject
• Missions of people involved
– Historians: spin compelling stories from facts
• Find symbolism, personality flaws and contrasts, etc.
– Education: use a story analogy
– Law: two competing stories on the same fabula
• Each side portrays a different victim
–
–
–
–
Politics: vilify the other side
Medical ethics: understand the patient’s story
Coincidences, conspiracy theories, police investigations
Oh… and art & entertainment
66
The almighty story
• Cognitive science: We even use story form in thinking
about and describing our own lives.
– We are all in a state of pursuing our wants
– Mental health “needs” are the same as our “wants”
• Explanations for actions such as joining gangs
– Diaries, gossip express selves through story
– Marketers appeal to our wants and promise a happy ending
67
Rule of Law
• We strongly fear a savage world
–
–
–
–
Loss of protection from civilization
Murder committed with impunity
We can be killed for no crime of our own
No deterrent of accountability
• Any threat to our rule of law is a villain, and any
protector is a hero we love and respect.
68
Rule of Law
• Backbone of entire genres of fiction:
– Western
• Sheriffs vs. outlaws in society on cusp of rule of law
– Post-apocalyptic
• Bandits, protectors of the peace
– Thrillers, spy movies
• Supervillains want to destroy order and accountability
• Serial killers impose fear by killing innocents
– Mafia
• Subculture with no rule of law
– War
69
Example: Rule of Law
• Common story in the news
– Remote places where there is no
rule of law
– Interviews with villagers living in
fear
– Terrorism
QuickTime™ and a
MPEG-4 Video decompressor
are needed to see this picture.
• Invoked for persuasion
– Aggrandize the heroes who will
protect us
– Cast the facts to support the story
as purely as possible
70
Modeling the world as stories
• We spread world views by casting people into
narrative roles.
–
–
–
–
Freedom fighter or terrorist?
Unreasonable nut or hero of reform?
Underdog or wannabe?
Careless doctor or victim of theft?
• Allegories and jokes as minimal models
– Small stories for abstracting and casting
– Powerful, but easily abused
71
Why wax philosophic?
• The more we look, the more we will see stories, and the
more we will see that how they overlap.
• The wider we look now, the better the model will be at
finding the core abstractions.
• We can’t explain the whole world.
– We can always condense and constrain to apply to some field.
72
The Plot
1.
2.
3.
4.
5.
A Word on Models
Narrative for News?
Related Work
The Big Idea
Waxing Philosophic
6. Back to News
7. Wrapping Up
73
Content selection
• Essential step for summarization
– Identify the main point of a document
– Separate from different types of details
• Sentence extraction with IR and other statistical
–
–
–
–
Inderjeet Mani
TF/IDF, word clustering, etc.
Radev, Teufel etc.
Probabilistic content models (Barzilay, Lee)
• Content selection techniques in generation
–
–
–
–
RST (Mann, Thompson, etc)
Discourse modeling (Marcu, Berstein, etc.)
Plans (Hovy, Moore, Paris, etc.)
Schemas (McKeown)
74
What is a story?
Text structure:
Pitch, elaboration, background (news)
Rising/falling action, climax, coda (fiction)
Discourse
Narrative semantics:
Heroes, villians, goals, obstacles, plot
arcs, themes, morals
Story
Raw timeline:
Actions, events, characters
Fabula
Observed
Implied
By modeling the top two layers, we might
better isolate the real-world events, and know
which are the most important for a summary.
75
Best place to look: The Pitch
JERUSALEM (CNN) -- The Israeli military has embarked on a
wide-ranging raid in Jenin after getting tips that the Palestinian
militant group Islamic Jihad was planning to carry out terror
attacks against Israel from the northern part of the West Bank,
the Israel Defense Forces said Thursday.
• Minimal summary of article
• Richest source for narrative insight
– Security theme, evacuation plot
– Can guide further content selection
• But… nontrivial to find
– Not always the lead sentence
76
Architecture
1.
Discourse Modeler
–
–
–
2.
Story Modeler
–
–
3.
Categorize each paragraph by function
Find the pitch
Separate the narrative news from non-narrative
Only so many stories can be told
From the pitch, find the “narrative backbone”
Plan Generator
–
–
Story-specific content selection
Story-specific generation plans or schemas
77
Architecture
Input
Text
1.
Summary
Discourse
Modeler
Discourse
– Categorize
each paragraph by function
modeler
Existing
– Find the pitch
Extraction,
– Separate the narrative news from non-narrative
Content Planning
Pitch
Lexicalization
2. Story Modeler
tools
– Only so many storiesStory
can be told
– From the pitch, findModeler
the “narrative backbone”
Plan
Narrative categories
3.
Plan Generator
–
–
Plan
Story-specific content selection
Story-specific generation plans Generator
or schemas
78
Discourse
modeler
• Reasonably well-defined structure for news articles
– Each paragraph serves a purpose
– Ordering is important
• Categorize each paragraph by function
Draft list:
• Pitch
• Elaboration
• Background
• Analysis
• MOS Reaction
• Character Description
• Testimonial
• Supportive
• Dissenting
• Expository
• Illustrative Anecdote
79
Simple declarative at beginning
BAGHDAD, Iraq (CNN) -- Saddam Hussein
stepped into an Iraqi court on Thursday
Pitch:
and entered“New
a new chapter
in Iraq's history, in
hearing
preliminary
charges against him
Development
Story
of
Conflict”
that included the gassing of Kurds and the invasion of Kuwait.
He listened to seven preliminary charges outlined in his arrest warrant -- the invasion
of Kuwait in 1990, suppression of the Kurdish and Shiite uprising in 1991, political
killings, religious killings, the killingBackground
of a big Kurdish family, and the gassing of the
Kurds in Halabja. List with dates
These are not the formal charges against Saddam, which will be worked out and
detailed in an indictment over the next
few months, beginning the investigative phase
Analysis
of the case.
The spectacle, meanwhile, was surprising for a generation of Iraqis who came to fear
MOS Reaction
the sight of him.
Saddam had bags under his eyes and looked gaunt. He had a neatly trimmed beard
and was thinner than he appeared
in December,
when he was seized in a hideout near
Character
Description
Tikrit.
• Different story types, include different types
of paragraphs, in different orders:
80
Quotations at beginning
"Mr. G.," aka chess instructor Douglas Goraleski, stood in front of a large cloth
chessboard hanging in the classroom and said to the 25 fifth-graders smiling back at him,
"Is it okay to cheat?"
Illustrative Anecdote
"You have to be a little sneaky," he confirmed, adding that playing chess also requires
patience, concentration, planning and strict adherence to the rules. And at risk of having
their game forfeited, the students did as they were told during their weekly hour of chess
instruction.
Oyster is one of hundreds of schools across the country where chess is being taught -not just as an after-school activity for students naturally drawn to the game, but during
school hours to youngsters of all abilities.
Pitch:
“New
Trend”
Story
In several U.S. cities and dozens
of countries,
the game
is being used more than ever as
a powerful educational tool. It helps develop skills ranging from critical thinking and time
management to sportsmanship and self-esteem, proponents have said.
"Everything that happens on a chessboard happens in real life," said Marley Kaplan,
president of Chess-in-the-Schools, Testimonial
a nonprofit program based in New York that teaches
chess to 38,000 inner-city elementary and middle school students.
Quotation
after pitch
•Different stories, different structures,
different priorities for summarization
81
Experiment: Methods
• Manual analysis to identify discriminatory features
• Hand-crafted decision tree
– Lexical, syntactic clues
– No notion of context
• Training/testing data: 350 lead article sentences
82
Training/Testing Data
• Labeled among pitch,
background, analysis,
testimonial
200
180
160
140
• Baseline: to assume
pitch = 52% accuracy
• Parsed (Collins 2000)
120
100
80
60
40
20
0
Pitch
Background
Analysis
Testimonial
83
Experiment: Algorithm
• Check for lexical and syntactic “blacklist” patterns
– Last, When, Recently = Background
– Question, noun phrase “IS” noun phrase = Analysis
– “X, said Y”; “X, Y told Z”; “Y told Z X” = Testimonial
• Match against patterns for simple declaratives
– No subject and past-tense action = Analysis
• Check for more patterns within the action clause
– Did, Going = Analysis
– Used to = Background
• Default = Pitch
84
Experiment: Results
Automatic
Analysis
Background
Pitch
Testimonial
Analysis
52
10
12
0
Background
2
17
2
1
Pitch
30
0
145
1
Testimonial
0
0
10
56
Truth
• 80% accuracy
• Can recurse on statements within testimonials
– Caveat: not yet labeled
85
Next step: Use the pitch
•
Classify the entire article
–
–
•
Narrative news: stories of conflict, new trends, revelations
Non-narrative news: analyses, editorials, reviews, etc.
If narrative, categorize the “backbone”
–
–
Applicable plots, characters, and themes
Statistically trained and classified
86
From “backbone” to summary
• Ordering and selection constraints
– “Always begin with the pitch”
– “Avoid anecdotes”
– “Elaboration before Testimonials”
• Different emphasis for different stories
– “History more important for Justice than Disaster,
Testimonials less important”
• New sentence generation
– Story types imply specific details (e.g., statistics)
– Lexicalization rules for expressing story types
– Story structure grammar
87
Next steps
• Apply other tools to discourse modeler
– Markov chains and HMMs for contextual clues
– Bayesian networks
– Decision tree learners
• Collect pitches and pick best narrative distinctions
– Experiment with methods for interpretation
• Finalize a summarization approach
88
The Plot
1.
2.
3.
4.
5.
6.
A Word on Models
Narrative for News?
Related Work
The Big Idea
Waxing Philosophic
Back to News
7. Wrapping Up
89
Denouement
• Model the real world with care
• The rabbit hole of story modeling
• Practical use for extractive summarization
– New categories to aid content selection
• Narrative news vs. non-narrative, etc.
– Discourse modeling, specialized for news
• Find the pitch, find the story
– Classification of narrative backbone
• Useful, yet practical (not full understanding)
90
Thank you!
91