Visual-Syntactic Text-Formatting:
Theoretical Basis and Empirical Evidence
for Impact on Human Reading
IEEE
International Professional Communication Conference
“Engineering the Future of Human Communication”
Seattle, October 1, 2007
Randall Walker, MD
Phil Schloss, BSEE, Adam Gordon, Charles Vogel, PhD
Charles R. Fletcher, PhD, Stan Walker, MD
WRT – Live Ink®
www.liveink.com
idea
idea
Language
Text
History
Physiology
Technology
Reading
Evolution of Speech and Text
Natural spoken language
50,000 years
Evolution of Speech and Text
Natural spoken language
50,000 years
Text
5000 years
Aristotle, “On Interpretation”
Spoken words are the symbols of mental
experience, and written words are the
symbols of spoken words.
Sumerian logographs from 4000 BC
SPKNWRDSRTHSMBLSFMNTLX
PRNCNDWRTTNWRDSRTHSMB
LSFSPKNWRDS.
2000 BC—Phoenician alphabet contained consonants only.
SPOKENWORDSARETHESYMBO
LSOFMENTALEXPERIENCEAND
WRITTENWORDSARTHESMYBO
LSOFSPOKENWORDS.
1000 BC—the Greeks added vowels.
SPOKENWORDSARETHESYMBOL
SOFMENTALEXPERIENCE,ANDW
RITTENWORDSARTHESMYBOLS
OFSPOKENWORDS.
200 BC—Aristophanes’ plays
with punctuation
Spokenwordsarethesymbolsofm
entalexperience,andwrittenword
sarethesmybolsofspokenwords.
700 AD—lower case
Spoken words are the symbols of
mental experience, and written
words are the symbols of spoken
words.
900 AD—spaces
between words.
1000 AD
The importance of word separation by
space is unquestionable, for it freed the
intellectual faculties of the reader, permitting
all texts to be read silently, that is, with the
eyes alone.
As a consequence, even readers of modest
intellectual capacity could read more swiftly,
and they could understand an increasing
number of inherently more difficult texts.
Paul Saenger:
Space Between Words:
The Origins of Silent Reading
Word separation, by altering the
neurophysiological process of reading,
simplified the act of reading, enabling both
the medieval and modern reader to receive
silently and simultaneously the text and
encoded information that facilitate both
comprehension and oral performance.
1000 AD
1500 AD
2000 AD
Reading, English & the Global Economy
• In US: over $1 trillion per year spent in
workers’ wages to read something
• Widening gap between grade level of
material and reading ability of workers,
including college graduates.
• Technical material often refractory to
editorial simplification or graphics.
• Consequences of inaccurate and inefficient
reading: accidents and injuries, wasted
materials, lost opportunities.
Reading, English & the Global Economy
• Non-native English speakers out-number
native English speakers 3-to-1.
• Within next decade, 3 billion people will
speak English; 2 billion will read it.
• Comprehension drops by at least 25
percent when reading technical material in
a non-native language (e.g., Scandinavian
physicians reading JAMA in English).
idea
idea
Language
Text
History
Physiology
Technology
Reading
The Reading Eye-Span:
Competition for Attention in Visual Processing
“A Sea of Words”
“One of the most challenging activities for the visual
attention mechanism in modern civilization.”
Squire Trelawney, Dr. Livesay, and the rest of these
gentlemen having asked me to write down the
whole particulars about Treasure Island, from the
beginning to the end, keeping nothing back but the
bearings of the island, and that only because there is
still treasure not yet lifted, I take up my pen in the
year of grace 17__ and go back to the time when
my father kept the Admiral Benbow inn and the
brown seaman with the sabre cut first took up his
lodging under one roof.
In Block Text – Minimal character-specific
information can be processed outside of the oval.
Everything else only competes for visual attention,
while adding no informational value.
Squire Trelawney, Dr. Livesay, and the rest of these
gentlemen having asked me to write down the
whole particulars about Treasure Island, from the
beginning to the end, keeping nothing back but the
bearings of the island, and that only because there is
still treasure not yet lifted, I take up my pen in the
year of grace 17__ and go back to the time when
my father kept the Admiral Benbow inn and the
brown seaman with the sabre cut first took up his
lodging under one roof.
Regressions = ~ 20% of the eye-movements
in reading
•At verbs or potential verbs
•After missing small, critical
function words needed for
disambiguating possible verbs.
•At phrase boundaries and new
sentences.
Regressions = ~ 20% of the eye-movements
in reading
•At verbs or potential verbs
•After missing small, critical
function words needed for
disambiguating possible verbs.
•At phrase boundaries and new
sentences.
Regressions reflect the
work of parsing.
Simple
Syntax
(joined
active
clause)
Complex
Syntax
(objectrelative
clause)
Vascular Responses to Syntactic Processing: Event-Related fMRI Study of Relative Clauses.
Caplan D, et al. Human Brain Mapping 15:26-30 (2001)
brought forth
a
our
fathers
Four score
and seven
years ago
new
conceived
nation
in liberty
and dedicated to
the proposition
upon this
continent
that
all men
are
created
equal
Human Speaker
Pitch:
High
Low
Stress
Elevated
Length
Rising
Rhythm
Falling
Concave
Cadence
Human Listener
In Speech, prosodic
attributes help the listener
parse the syntax of the
spoken sentence.
These prosodic-syntactic
cues operate in real-time,
word-by-word.
The cues are powerful
enough to enable a listener
to predict the part-of-speech
of not-yet-heard phrases.
Dynamic Collaboration
between lexical and syntactic processing
Cerebral Cortex, Vol. 11, No. 3, 223-237, March 2001
The Neural Bases of Sentence Comprehension: a fMRI Examination of Syntactic and Lexical
Processing.
Timothy A. Keller, Patricia A. Carpenter and Marcel Adam Just
Center for Cognitive Brain Imaging, Carnegie Mellon University
1000 AD
1500 AD
2000 AD
idea
idea
Language
Text
History
Physiology
Technology
Reading
Spoken words are the symbols of
mental experience, and written
words are the symbols of spoken
words.
900 AD—spaces
between words.
Spoken words
are the symbols
of mental experience,
and written words
are the symbols
of spoken words.
2000 AD
LiveInk®
When in the Course
of human events,
it
becomes necessary
for one people
to dissolve
the political bands which
have connected them with another,
and
to assume
among the powers
of the earth,
the separate and equal station
to which the Laws of Nature
and
of Nature's God
entitle them,
a decent respect
to the opinions of mankind
requires that
they
should declare
the causes which
impel them to the separation.
When in the Course of human events it
becomes necessary for one people to
dissolve the political bands which have
connected them with another and to
assume among the powers of the
earth, the separate and equal station
to which the Laws of Nature and of
Nature's God entitle them, a decent
respect to the opinions of mankind
requires that they should declare the
causes which impel them to the
separation.
Millions of
computer
calculations per
sentence
US Dept of Education Innovation Research Awards: 2001-2004
Visual-Syntactic Cueing
to Improve Human Reading
Visual Processing
and Block Text
Cognitive Work of
Complex Syntax
Simple
Syntax
Complex
Syntax
Evidence for Dynamic Collaboration between Sub-processes
Unit Exams: 10th Grade History
Live Ink
Control
85
80
75
70
65
60
1
2
3
4
5
6
7
8
Test of Repeated Measures: F1,83=2151, p<.001
9
10
Proportion of Students with Score
10th Grade Final Exam
mean +/- 2 S.D.
Control
30
40
50
60
70
80
90
100
Proportion of Students with Score
10th Grade Final Exam
mean +/- 2 S.D.
VSTF
Control
30
40
50
60
70
80
90
100
Improved Reading Proficiency
in Non-native English Readers (ESOL)
230
Control
EL1
220
Live Ink
EL1
210
Control
ESOL
200
Live Ink
ESOL
190
Fall
Grade 6
Spring
9:1 Preference over Block Format
Live Ink
Uncertain
Block
260 High School students,
after 1 year of use
When in the Course
of human events,
it
becomes necessary
for one people
to dissolve
the political bands which
have connected them with another,
and
to assume
among the powers
of the earth,
the separate and equal station
to which the Laws of Nature
and
of Nature's God
entitle them,
a decent respect
to the opinions of mankind
requires that
they
should declare
the causes which
impel them to the separation.
Text fabrication, for
Alternative View of
Linear Texts
VSTF
Transformation
Engine
User-directed,
On-the-Fly
Parsing Service
When in the Course of human events it
becomes necessary for one people to
dissolve the political bands which have
connected them with another and to
assume among the powers of the
earth, the separate and equal station
to which the Laws of Nature and of
Nature's God entitle them, a decent
respect to the opinions of mankind
requires that they should declare the
causes which impel them to the
separation.
System-integrated Text
Enrichment with Userbased Manipulation
Try Online Demo at:
www.nortonebooks.com
Click on Live Ink® feather.
Advance through
Live Ink panels
1000 AD
The importance of word separation by
space is unquestionable, for it freed the
intellectual faculties of the reader, permitting
all texts to be read silently, that is, with the
eyes alone.
As a consequence, even readers of modest
intellectual capacity could read more swiftly,
and they could understand an increasing
number of inherently more difficult texts.
Paul Saenger:
Space Between Words:
The Origins of Silent Reading
Word separation, by altering the
neurophysiological process of reading,
simplified the act of reading, enabling both
the medieval and modern reader to receive
silently and simultaneously the text and
encoded information that facilitate both
comprehension and oral performance.
Negroponte: “Being Digital”
"By looking for
the structure in signals,
how they were generated,
we go
beyond the surface appearance
of bits
and
discover the building blocks
out of which the image,
sound,
or text came.
This is one
of the most important facts
of digital life."
“By looking for the structure in
signals, how they were
generated, we go beyond the
surface appearance of bits
and discover the building
blocks out of which the image,
sound, or text came. This is
one of the most important
facts of digital life.”
TheNextText