Unstructured Data and the Role
of Natural Language Processing
Philip Resnik
Department of Linguistics and
Institute for Advanced Computer Studies
University of Maryland
Wolfram Data Summit
September 7, 2012
♬ Daisy, Daisy…
Hiya, guys. What did
you think of Obama’s
speech last night? I
think I liked
Michelle’s better.
Beep.
I didn’t watch it. I
was playing a nice
game of chess.
2
3
The statistical revolution in NLP
% “Statistical”
Papers
100%
80%
60%
40%
20%
0%
2005
2000
1995
1990
1985
AI Winter
Annual Meeting of the Association for
Computational Linguistics
Sources: graph adapted from Church, K. (2003) “Speech and Language Processing: Where have we been and where are we going,” Eurospeech,
Geneva, Switzerland. Green circle data have been added from figures in Cardie and Mooney (1999).
4
NLP is no longer about getting
machines to understand
language like people do.
It’s about building machines that
do things with language that
people find useful.
5
Surface methods
6
Surface methods plus categories
Pennebaker, Linguistic Inquiry and Word Count
http://www.liwc.net/tryonlineresults.php
7
Surface methods plus categories
Brendan O’Connor, Ramnath
Balasubramanyan, Bryan R.
Routledge, Noah A. Smith, From
Tweets to Polls: Linking Text
Sentiment to Public Opinion
Time Series, Proceedings of the
International AAAI Conference
on Weblogs and Social Media,
Washington, DC, May 2010.
8
Surface methods plus categories
*Note that Noah Smith did point out this ambiguity!
Surface methods plus hidden structure
natuerlich hat john spass am spiel
of course john has fun with the game
http://www.statmt.org/moses/?n=Moses.Background
10
One morning I shot an
elephant in my pajamas.
How he got in my pajamas, I
don’t know.
One morning I shot an elephant in my pajamas.
11
Extracting structured information
HPI: Atrial fibrillation. This patient is a 56-year-old white gentleman who has had a
history of atrial fib on and off since he had his bypass surgery. Patient was originally
diagnosed with coronary artery disease as well as mitral valve problems approximately
3 years ago. Dr. Tirona used to take care of him at that time. He had a bypass surgery
as well as mitral valve repair done at that time. Postop he had an episode of A-fib
which then resolved spontaneously. He remembers somebody talking to him about
cardioversion, but then the A-fib resolved spontaneously. So he was started on
Coumadin. He would get some occasional episodes, but usually they are very brief, so
he never bothered about them. Of late, over the last few months, he has been getting
more frequent episodes and duration of these episodes is also prolonged for a few
hours. So he saw Dr. Hagan who has referred him here for further evaluation and
treatment. The patient states when he does get the A-fib, he feels very weak, tired, and
short of breath. He denies any chest pain. Otherwise he is usually very active
physically, he works fulltime as an electrician, and has not had any problems as far as
doing his day-to-day work.
MEDICAL HISTORY: 1. Coronary artery disease as mentioned above. 2.
Hypertension. 3. Hypercholesterolemia.
.
IMPRESSION: Paroxysmal atrial fibrillation in a patient with prior mitral valve disease,
currently having more frequent breakthroughs symptoms.
Extracting structured information
Type
HPI: Atrial fibrillation. This patient is a 56-year-old white gentleman who has had a
history of atrial fib on and off since he had his bypass surgery. Patient was originally
diagnosed with coronary artery disease as well as mitral valve problems approximately
3 years ago. Dr. Tirona used to take care of him at that time. He had a bypass surgery
as well as mitral valve repair done at that time. Postop he had an episode of A-fib
which then resolved spontaneously. He remembers somebody talking to him about
cardioversion, but then the A-fib resolved spontaneously. So he was started on
Coumadin. He would get some occasional episodes, but usually they are very brief, so
he never bothered about them. Of late, over the last few months, he has been getting
more frequent episodes and duration of these episodes is also prolonged for a few
hours. So he saw Dr. Hagan who has referred him here for further evaluation and
treatment. The patient states when he does get the A-fib, he feels very weak, tired, and
short of breath. He denies any chest pain. Otherwise he is usually very active
physically, he works fulltime as an electrician, and has not had any problems as far as
doing his day-to-day work.
MEDICAL HISTORY: 1. Coronary artery disease as mentioned above. 2.
Hypertension. 3. Hypercholesterolemia.
.
IMPRESSION: Paroxysmal atrial fibrillation in a patient with prior mitral valve disease,
currently having more frequent breakthroughs symptoms.
Anatomy
Modifier
Diagnosis/Problem
-
atrial
-
fibrillation
history
atrial
-
fibrillation
-
mitral_valve
-
problem
-
-
-
weak
-
-
-
tired
-
-
-
short_of_breath
-
disease
history coronary_artery
history
-
-
hypertension
-
-
-
hypercholesterolemia
-
atrial
history mitral_valve
paroxysmal fibrillation
-
disease
Extracting structured information
ICD
Type
HPI: Atrial fibrillation. This patient is a 56-year-old white gentleman who has had a
history of atrial fib on and off since he had his bypass surgery. Patient was originally
diagnosed with coronary artery disease as well as mitral valve problems approximately
3 years ago. Dr. Tirona used to take care of him at that time. He had a bypass surgery
as well as mitral valve repair done at that time. Postop he had an episode of A-fib
which then resolved spontaneously. He remembers somebody talking to him about
cardioversion, but then the A-fib resolved spontaneously. So he was started on
Coumadin. He would get some occasional episodes, but usually they are very brief, so
he never bothered about them. Of late, over the last few months, he has been getting
more frequent episodes and duration of these episodes is also prolonged for a few
hours. So he saw Dr. Hagan who has referred him here for further evaluation and
treatment. The patient states when he does get the A-fib, he feels very weak, tired, and
short of breath. He denies any chest pain. Otherwise he is usually very active
physically, he works fulltime as an electrician, and has not had any problems as far as
doing his day-to-day work.
MEDICAL HISTORY: 1. Coronary artery disease as mentioned above. 2.
Hypertension. 3. Hypercholesterolemia.
.
IMPRESSION: Paroxysmal atrial fibrillation in a patient with prior mitral valve disease,
currently having more frequent breakthroughs symptoms.
Anatomy
Modifier
Diagnosis/Problem
394.9 atrial 427.31
fibrillation
Other and
unspecified
mitral
fibrillation
atrial 427.31
history
valve diseases
-
mitral_valve
-
- 780.79
427.31
- 780.79
Atrial fibrillation
-
-
history coronary_artery
history
-
-
-
-
-
atrial
history mitral_valve
problem
weak
tired
786.05
short_of_breath
414.01
-
disease
401.9
272.0
-
paroxysmal
427.31
-
394.9
hypertension
hypercholesterolemia
fibrillation
disease
To react to this talk on your phone or
laptop while you listen: visit
go.reactlabs.org and select #EHR.
15
Medical coders without NLP
Medical coders with NLP
Dr. Martin S. Kohn | Clinical Decision Support: DeepQA,
http://www.im.org/Meetings/Past/2012/2012APMWinterMeeting/Documents/Winter%20Meeting%20Presentations/P
17
Discovering structure
05_03_02.txt.0002
BEGALA Good evening. Welcome to CROSSFIRE,
05_03_02.txt.0002
evening.
Welcomeinto
CROSSFIRE,
coming
to you live from BEGALA
the GeorgeGood
Washington
University
beautiful
05_03_02.txt.0002
BEGALA
Welcome
CROSSFIRE,
coming to
you live from
theTonight
GeorgeGood
Washington
University
into
beautiful
downtown
Washington,
D.C.
in
the evening.
CROSSFIRE,
the case
of
coming
to
you
live
from
the
George
Washington
University
in beautiful
downtown
Washington,
D.C.
Tonight
in
the
CROSSFIRE,
the
case
of
the Reverend Paul Shanley, the Roman Catholic priest facing child
downtown
Washington,
D.C.
Tonight
in
the
CROSSFIRE,
the
case
of
the
Reverend
Paul
Shanley,
the
Roman
Catholic
priest
facing
child
rape charges in Massachusetts. Should his superiors be held
thecharges
Reverend
Shanley,
Romanof
Catholic
priest
facing
rape
in Paul
Massachusetts.
Should
his
superiors
be
held child
responsible?
Also,
Matt
Drudge,the
founder
the
Internet
"Drudge
rape charges
in Massachusetts.
Should
histhe
superiors
be
responsible?
Also,
Matt muckraker,
Drudge, founder
of
Internetor
"Drudge
Report."
Is he a right-wing
an Internet
gossip
aheld
responsible?
Also,
Matt
Drudge,
ofwhen
the gossip
Internet
"Drudge
Report."
Is he a right-wing
muckraker,
an Internet
a
legitimate
journalist?
We'll
ask
Drudgefounder
himself
we getor
him
Report."
Is
he
a
right-wing
muckraker,
an
Internet
gossip
a
legitimate
journalist?
We'll
ask
Drudge
himself
when
we
get
him
in the CROSSFIRE. First, flying the not-so-friendly skies, wouldor
legitimate
journalist?
We'll
ask
Drudge
himself
when
we
get
him
in
the
CROSSFIRE.
First,
flying
the
not-so-friendly
skies,
would
you feel safer if pilots were armed? One outspoken congressional
inisfeel
the
CROSSFIRE.
First,
flying
theOne
not-so-friendly
skies,
you
safer having
if pilotsguns
werein
armed?
outspoken
congressional
critic
against
the cockpit.
We're going
to would
youisfeel
if Please
pilotsguns
were
One outspoken
congressional
critic
against
inarmed?
the cockpit.
We're going
to the
introduce
hersafer
now.having
welcome,
Eleanor
Holmes
Norton,
critic is delegate
against
guns
in theofcockpit.
We're
to thank
introduce
her now.having
Please
welcome,
Eleanor
Holmes
Norton,
the
Democratic
from
the
District
Columbia.
Ms.going
Norton,
introduce
her
now.
Please
welcome,
Eleanor
Holmes
Norton,
the
Democratic
delegate
from
the
District
of
Columbia.
Ms.
Norton,
thank
you. Welcome back.
Democratic
you.
Welcomedelegate
back. from the District of Columbia. Ms. Norton, thank
you. Welcome back.
05_03_02.txt.0003
CARLSON Now, Ms. Norton, the majority,
CARLSON
Now,
Ms.are
Norton,
the majority,
the05_03_02.txt.0003
vast majority of commercial
airline
pilots
strongly
in favor
05_03_02.txt.0003
CARLSON
Now,
Ms.airliners.
Norton,
the
majority,
vast majority
airline
pilots
are
strongly
in favor
of the
carrying
guns in of
thecommercial
cockpit
on commercial
You're
the
vast
majority
of
commercial
airline
pilots
are
strongly
of
carrying
guns
in
the
cockpit
on
commercial
airliners.
You're
against it. What do you as a delegate know about operating
a in favor
of
carrying
guns
in
the
cockpit
on
commercial
airliners.
You're
against
it.
What
do
you
as
a
delegate
know
about
operating
a
commercial airliner that the majority of commercial airline pilots
against
What do
you
asmajority
a delegate
know aboutairline
operating
a
commercial
that
the
of commercial
pilots
don't
know it. airliner
commercial
airliner that the majority of commercial airline pilots
don't
know
don't
know
05_03_02.txt.0004
DELEGATE
Well, I know what
05_03_02.txt.0004
Well,
what what
Transportation
SecretaryDELEGATE
Norm Mineta tells
me,I know
and I know
05_03_02.txt.0004
DELEGATE
Well,
I know
what
Transportation
Secretary
Norm
Mineta
tells
me,
and
I know
what
Homeland Security Adviser Tom Ridge tells
me,
and
they
are against
Transportation
Secretary
Mineta
me,
and
I know
what
Adviser
Tomagainst
Ridge
tells
me,
and
they
are against
it. Homeland
And
I thinkSecurity
the reason
they Norm
are
it tells
is you
don't
want
Homeland
Security
Tom
Ridge tells
me,
and
they
are
Andwho's
I thinkflying
the reason
it up
is you
don't
want
theit.guy
oneAdviser
of they
theseare
bigagainst
busters
there
also
with
a against
it.
And
I
think
the
reason
they
are
against
it
is
you
don't
want
theinguy
flying to
one
of these
busters
there
with a
gun
his who's
hand trying
protect
hisbig
plane.
Youup
want
air also
marshals
the
guy
who's
flying
one
of
these
big
busters
up
there
also
with
gun
in
his
hand
trying
to
protect
his
plane.
You
want
air
marshals
to do that. You want flight attendants to understand how to protect a
gun
in his
hand
trying
toattendants
protect
histo
plane.
do that.
You
want
flight
understand
howair
tomarshals
protect
thetocockpit.
And
you
want
the
redundancies
thatYou
we want
have
built
in,
do that.
You
wantwant
flight
attendants
understand
howbuilt
to protect
thetocockpit.
And
you
the
redundancies
that
in,
redundancy
after
redundancy,
working
fortoyou.
We we
arehave
panicking
the
the cockpit.
And
yousay,
want
redundancies
we
have
redundancy
after
redundancy,
working
you.that
We
are
panicking
the
American
people.
They
ohthe
my
God, Ifor
thought
they
had
thebuilt in,
redundancy
after
redundancy,
working
for
you.
We
are
panicking
the
American
people.they
They
say,
ohHere
my God,
I thought
they
had the
hearings,
I thought
did
that.
come
the pilots
saying,
American
people.
They
say,
oh
my
God,
I
thought
they
had
the
hearings,
I
thought
they
did
that.
Here
come
the
pilots
saying,
oh no, they haven't. We've got to have guns.
I thoughtWe've
they did
Here
come the pilots saying,
ohhearings,
no, they haven't.
gotthat.
to have
guns.
oh no, they haven't. We've got to have guns.
Looking at just word
counts often gives you
a mish-mash.18
Discovering structure
05_03_02.txt.0002
BEGALA Good evening. Welcome to CROSSFIRE,
05_03_02.txt.0002
evening.
Welcomeinto
CROSSFIRE,
coming
to you live from BEGALA
the GeorgeGood
Washington
University
beautiful
05_03_02.txt.0002
BEGALA
Welcome
CROSSFIRE,
coming to
you live from
theTonight
GeorgeGood
Washington
University
into
beautiful
downtown
Washington,
D.C.
in
the evening.
CROSSFIRE,
the case
of
coming
to
you
live
from
the
George
Washington
University
in beautiful
downtown
Washington,
D.C.
Tonight
in
the
CROSSFIRE,
the
case
of
the Reverend Paul Shanley, the Roman Catholic priest facing child
D.C.
Tonight
in the
CROSSFIRE,
case of
thedowntown
Reverend
Paul
Shanley,
theShould
Romanhis
Catholic
priest
rape
charges
inWashington,
Massachusetts.
superiors
befacing
heldthechild
thecharges
Reverend
Shanley,
Romanof
Catholic
priest
facing
rape
in Paul
Massachusetts.
Should
his
superiors
be
held child
responsible?
Also,
Matt
Drudge,the
founder
the
Internet
"Drudge
rape charges
in Massachusetts.
Should
histhe
superiors
be
responsible?
Also,
Matt muckraker,
Drudge, founder
of
Internetor
"Drudge
Report."
Is he a right-wing
an Internet
gossip
aheld
responsible?
Also,
Matt
Drudge,
founder
of
the
Internet
"Drudge
Report."journalist?
Is he a right-wing
muckraker,
an Internet
a
legitimate
We'll ask
Drudge himself
whengossip
we getor
him
Report."
Is he aFirst,
right-wing
muckraker,
an Internet
gossip
a
journalist?
We'll
ask
Drudge
himself
when
we
getor
him
in legitimate
the
CROSSFIRE.
flying
the
not-so-friendly
skies,
would
ask
himself when
get him
in legitimate
the CROSSFIRE.
flying
theDrudge
not-so-friendly
skies,we
would
you
feel
safer ifjournalist?
pilotsFirst,
wereWe'll
armed?
One
outspoken
congressional
inisfeel
the
CROSSFIRE.
First,
flying
theOne
not-so-friendly
skies,
you
safer having
if pilotsguns
werein
armed?
outspoken
congressional
critic
against
the cockpit.
We're going
to would
youisfeel
if Please
pilotsguns
were
One outspoken
congressional
critic
against
inarmed?
the cockpit.
We're going
to the
introduce
hersafer
now.having
welcome,
Eleanor
Holmes
Norton,
critic
is
against
having
guns
in
the
cockpit.
We're
going
to thank
introduce delegate
her now. from
Please
Holmes
Norton,
the
Democratic
thewelcome,
District ofEleanor
Columbia.
Ms. Norton,
introduce
her
now.
Please
welcome,
Eleanor
Holmes
Norton,
the
Democratic
delegate
from
the
District
of
Columbia.
Ms.
Norton,
thank
you. Welcome back.
Democratic
you.
Welcomedelegate
back. from the District of Columbia. Ms. Norton, thank
you. Welcome back.
05_03_02.txt.0003
CARLSON Now, Ms. Norton, the majority,
CARLSON
Now,
Ms.are
Norton,
the majority,
the05_03_02.txt.0003
vast majority of commercial
airline
pilots
strongly
in favor
05_03_02.txt.0003
CARLSON
Now,
Ms.airliners.
Norton,
the
majority,
vast majority
airline
pilots
are
strongly
in favor
of the
carrying
guns in of
thecommercial
cockpit
on commercial
You're
the
vast
majority
of
commercial
airline
pilots
are
strongly
of
carrying
guns
in
the
cockpit
on
commercial
airliners.
You're
against it. What do you as a delegate know about operating
a in favor
of carrying
guns
in
the
cockpit
on
You're
against
it. airliner
What
do
you
as
a delegate
know aboutairliners.
operating
a
commercial
that
the
majority
ofcommercial
commercial
airline
pilots
against
What do
you
asmajority
a delegate
know aboutairline
operating
a
commercial
that
the
of commercial
pilots
don't
know it. airliner
commercial
airliner
that
the
majority
of
commercial
airline
pilots
don't know
don't
know
05_03_02.txt.0004
DELEGATE
Well, I know what
05_03_02.txt.0004
Well,
what what
Transportation
SecretaryDELEGATE
Norm Mineta tells
me,I know
and I know
05_03_02.txt.0004
DELEGATE
Well,
I know
what
Transportation
Secretary
Norm
Mineta
tells
me,
and
I know
what
Homeland
Security
Adviser
Tom Ridge
tells
me,
and
they
are against
Transportation
Secretary
Mineta
me,
and
I know
what
Adviser
Tomagainst
Ridge
tells
me,
and
they
are against
it. Homeland
And
I thinkSecurity
the reason
they Norm
are
it tells
is you
don't
want
Homeland
Security
Adviser
Tom
Ridge
tells
me,
and
they
are
Andwho's
I thinkflying
the reason
is you
don't
theit.guy
one of they
theseare
bigagainst
bustersit up
there
alsowant
with
a against
it.
And
I
think
the
reason
they
are
against
it
is
you
don't
want
theinguy
flying to
one
of these
busters
there
with a
gun
his who's
hand trying
protect
hisbig
plane.
Youup
want
air also
marshals
who's
flying
one
of these
big
busters
there
with a
inguy
his
hand
trying
to
protect
histo
plane.
Youup
want
air
marshals
to gun
dothe
that.
You
want
flight
attendants
understand
how
toalso
protect
gun
in his
hand
trying
toattendants
protect
histo
plane.
do that.
You
want
flight
understand
howair
tomarshals
protect
thetocockpit.
And
you
want
the
redundancies
thatYou
we want
have
built
in,
do that.
You
wantwant
flight
attendants
understand
howbuilt
to protect
thetocockpit.
And
you
the
redundancies
that
in,
redundancy
after
redundancy,
working
fortoyou.
We we
arehave
panicking
the
the cockpit.
And
yousay,
want
redundancies
we
have
redundancy
after
redundancy,
working
you.that
We
are
panicking
the
American
people.
They
ohthe
my
God, Ifor
thought
they
had
thebuilt in,
redundancy
after
redundancy,
working
for
you.
We
are
panicking
the
American
people.they
They
say,
ohHere
my God,
I thought
they
had the
hearings,
I thought
did
that.
come
the pilots
saying,
American
people.
They
say,
oh
my
God,
I
thought
they
had
the
hearings,
I
thought
they
did
that.
Here
come
the
pilots
saying,
oh no, they haven't. We've got to have guns.
I thoughtWe've
they did
Here
come the pilots saying,
ohhearings,
no, they haven't.
gotthat.
to have
guns.
oh no, they haven't. We've got to have guns.
*Wikipedia: Topic Model; Blei et al. 2003
Bayesian topic models*
discover the distinct topics
interwoven in documents.
19
Model: detecting topic shifts
Model: detecting topic shifts
21
Model: topic shift tendency
22
Ifill, moderator: Terrible. Yes, she was constrained by
the agreed debate rules. But she gave not the slightest
sign of chafing against them or looking for ways to
follow up the many unanswered questions or selfcontradictory answers. This was the big news of the
evening. Katie Couric, and for that matter Jim Lehrer,
have never looked so good.
23
Model: topic shift tendency
24
Take-aways
• The role of NLP is not “understanding”. It’s
helping people do useful things with
language.
• Shallow methods work extremely well…
except when they don’t. Language is
replete with underlying structure.
• The deep value to look for in NLP is in
bringing that structure to the surface and
making it accessible to human insight.
25
Thanks!
26
reactlabs.org
Four years ago, I know that many Americans felt a fresh
excitement about the possibilities of a new president. That
choice was not the choice of our party, but Americans always
come together after elections. We're a good and generous
people, and we are united by so much more than what
divides us. When that election was over, when the yard
signs came down and the television commercials finally
came off the air, Americans were eager to go back to work,
to live our lives the way Americans always have,
optimistic and positive and confident in the future. That
very optimism is uniquely American. It's what brought
us to America. We're a nation of immigrants, we're the
children and grandchildren and great-grandchildren of
the ones who wanted a better life. The driven ones. The
ones who woke up at night, hearing that voice telling
them that life in a place called America could be better.
They came, not just in pursuit of the riches of this world,
but for the richness of this life.
Take-aways
• The role of NLP is not “understanding”. It’s
helping people do useful things with
language.
• Shallow methods work extremely well…
except when they don’t. Language is
replete with underlying structure.
• The deep value to look for in NLP is in
bringing that structure to the surface and
making it accessible to human insight.
31
Thanks!
32
33
34
"In this 10-year time frame, . . .
we will have perfected speech
recognition and speech output
well enough that those will
become a standard part of the
interface."
Bill Gates, 1997
35