Learning with audio-visible
and invisible authors
Lessons learnt from a Political science course
at USI – University of Lugano, Switzerland
Terry Inglese
UCSB, 14th November 2006
My background
• TV program designer and archivist at the Swiss public
TV
• Indipendent documentary filmmaker
• PhD student
• Multimedia
instructional designer
Combination of
academic and professional
skills
The memory institutions and the digital revolution
• Archives - from storing objects to the
management of the life cycle of digital and
digitized products;
• Libraries - from the reading room to the
digital information service centre;
• Museums - from collections to narrative
connections and new experiences.
• from a static into a dynamic change
Database and Narrative
(Manovich – 2001)
• “The emergence of new media coincides
… with accessing and reusing existing
media objects as with creating new ones.”
• (Manovich, 2001, pp. 35, 36)
TV archive and new media
• New media tend to repackage old media through
reproduction, manipulation, convergence.
• Benjamin (1977) and Baudrillard (1996)
– the loss of aura that the original work of art would lose
if inserted into the age of mechanical reproduction.
• In our case studies, besides an apparent feeling
of loss, there is a sense of gain that new media
are providing, a gain that is not material, but
cognitive and affective.
The auratic value of new media
• New media are not just invented to meet the needs that
already exist, but also new ones.
• Kress and Van Leeuwen, “what is lost may return, and
what is gained many yet turn out to be loss. The new
technologies’ emphasis on multimodality, three
dimensionality and interactivity can be seen as a
return of many of the things that were lost in the
transition from orality to literacy, as a second orality, in
other words.” (Kress, Van Leeuwen, 2001, p. 92).
• New media possess an auratic value, as the one of the
medieval manuscripts, because these media become
manuscript media, stored, transmitted, encoded and reencoded in a digital form.
The conceptual orality
• From this perspective, archives provide a return
to a conceptual orality, that is to say, “a return to
the medieval framework wherein words or
documents gained meaning only as they were
closely related to their context and to actions
arising from that context. In that oral tradition,
meaning lies not in the records themselves, but
(in) the transactions and customs to which they
borrow witness as ‘evidences.’ (...).”
• (Taylor, in Cook, Dodds, 2003, p. 23).
Taylor
• “a catalogue remains a catalogue, but
pattern recognition is the chef-d’oeuvre of
human intelligence”.
The added value:
• Discovering patterns of knowledge
• Extracting more from less
(Taylor, in Cook, Dodds, 2003 )
My work
Goal:
• to recognize the pedagogical as well as
instructional benefits of studying with TV
multimedia archives as learning tools, taking
into account the students’ perspective.
• The genre: TV interviews with Social Science
scholars usually read and studied only through
texts.
RTSI – Swiss public TV and radio
• Radio – RSI - 30ies
• TV – RTSI - 60ies
• because of the lack of the higher
education institution like a university, the
RTSI produced highly cultural programs
(TV and radio) enhancing education and
culture (Inglese, 1998)
• Genre of TV program – the interview
Availability and Affordances concepts
availability concept: archival content, produced in the
past with analogue technologies, is easily ‘transformed’
into digital support. This technological shift increases
the potential availability of ‘multimedia contents’;
affordance concept: students learn more deeply when
they feel a personal relationship with the author to be
studied, if the author (hypothesis) is offered – beside
the text - through a multimedia & multimodal format
such as a TV interview.
(Inglese, Mayer, Rigotti (2007, in press))
Media for learning
• Media can be defined by:
1. the technology
2. the symbol systems
3. the processing capabilities
media attributes and instructional methods
• it is not possible to separate the media attributes
from the instructional methods
• “What we have learned from all the media
comparison research is that it’s not the medium,
but rather the instructional methods that cause
learning.” (Mayer, Clark, 2003, p. 21)
• Learning depends on the quality of instructional
messages and instructional methods rather than
the medium per se.
Multimedia learning - Mayer
• The cognitive theory of multimedia learning. (2001, 2003, 2005, etc.)
• Multimedia:
• presentation of material using both words and pictures.
• Multimedia learning is “a sense-making activity in which the learner
seeks to build a coherent mental representation from the presented
material.”
• “the learner’s job is to make sense of the presented material; thus,
the learner is an active sense maker who experiences a multimedia
presentation and tries to organize and integrate the presented
material into a coherent mental representation.” (Mayer, 2001, p. 1315)
• The cognitive theory of multimedia learning is consistent with
empirical research results based on how people learn and the
understanding of the main cognitive processes in learning.
Multimedia learning
Social agency theory - Mayer
• assumptions that social cues in multimedia instructional messages
prime a social response in learners
• the learner can interpret a multimedia instructional message as
– an instance of information delivery or/and
– a social communication event.
Influencing
the type of schemas activated in the learner
the type of cognitive processing, and
the quality of the learning outcomes
– Multimedia principle (Mayer, 2001)
– Personalization principle (Mayer, 2003) – I-you vs. 3rd person
– Sound cues – voice and the concept of persona (Mayer, 2005,
2006)
‘the media equation paradigm’.
• people apply the same dynamics from human-human
interactions to human-computer interactions.
• Reeves and Nass (1996) define this phenomenon ‘the
media equation paradigm’. This paradigm states that the
media equal real life.
• All people automatically and unconsciously respond
socially to media, and therefore also to computer and
television, … especially nowadays with the new and old
media convergence.
• Media features: closeness, size, visual and audio fidelity,
motion, memory, and voice.
Research questions
• Would students learn (read and write) better if teachers
could use texts and audiovisual materials, such as TV
archived interviews with scholars that are normally
studied through texts only?
– retention and motivation (did students retain and enjoy more with
an audio-visible author vs. an invisible text-based author?)
• Grade
• Number of written words
• Quote
• Does reading text of scholars offered through multimedia
instructional messages change? If yes, how and why?
• the social presence of the author
• Kamin, Intrator and Kim (2000) have
shown despite the appeal of using
multimedia documents to enhance
reading, there are still several
cognitive and affective processes in
reading and processing texts that
need to be investigated.
The case study - Definitions
• Audio-visible author
– Multimedia and multimodal interview format
– I-you relationship
– & Text
• Invisible author
– Only text
– No I-you relationship
– Detached, formal and academic language
(strong text)
1. Case study
The final written exam texts
• 105 students took the written exam
• 73 native Italian speakers and 32 non-native ones.
• The non-native Italian students could write it in their own
languages: German, French, Spanish or English.
• 7 questions:
– 2 questions related to the audio-visible authors and
– 1 related to the invisible author.
• 10-to-1 grading scale, with 10 as the highest grade and 1
as the lowest.
• Karl Popper – audio-visible author + text
• Paul Feyerabend – audio-visible + text
• Andrea Semprini – invisible author – only
text (a text that is invisible)
grades
7.8
7.67
7.53
7.6
7.4
7.2
7
7
7
mean grades Ita
6.81
6.8
6.67
6.6
6.4
6.2
6
Feyerabend - vis
Popper - vis
Semprini - invis
mean grades non Ita
words
300
250
200
246
194
166
146
150
133
143
mean words Ita
mean words non Ita
100
50
0
Feyerabend - vis
Popper - vis
Semprini - invis
quotes
3
2.41
2.5
2.03
2
1.6
1.5
means quotes Ita
1.5
means quotes non Ita
1
0.26 0.31
0.5
0
Feyerabend - vis
Popper - vis
Semprini - invis
2. Case study
• 108 university freshmen
• 60 native Italian speakers and 48 nonnative ones (15 different countries and
speaking a total of 13 different languages).
• the average age in the class was from 20
to 25 years old
• Claude Lévi-Strauss – audio-visible author
+ text
• Paul Feyerabend – audio-visible author +
text
• Andrea Semprini – invisible author
grades
9
8
7.98 7.78
7.99
7.04
7
7.17
5.8
6
mean grades Ita
5
4
mean grades non Ita
3
2
1
0
Feyerabend - vis Lévi Strauss vis
Semprini - invis
words
200
180
160
140
120
100
80
60
40
20
0
173
150 155
158
161
112
mean words Ita
mean words non Ita
Feyerabend vis
Lévi Strauss - Semprini - invis
vis
quotes
3.5
3.28
3
3
2.57
2.75
2.5
2
means quotes Ita
means quotes non Ita
1.5
1
0.61
0.33
0.5
0
Feyerabend vis
Lévi Strauss - Semprini - invis
vis
Qualitative data
• thinking and feeling aloud methodology to collect the
ongoing cognitive and affective responses and
processes of students while reading the texts aloud.
•
•
•
•
After reading, three questions:
1. which is the most comprehensible author and why?
2. which is the most interesting author and why?
3. which is the most closer author and why?
• 1. to imagine how the invisible author looks like
• 2. Bus scenario
• 3. Taxi scenario
Measuring the author‘s presence in
their texts
Levi Strauss (162 Words)
I
0%
you
0%
we
1.90%
questions
0%
exam ples
0%
Total
1.90%
Feyerabend (258 Words)
0.40%
0%
1.90%
0.80%
0.80%
3.90%
Sem prini (197 Words)
0%
0%
1%
0%
0%
1%
Taxi
28 Italian and non Italian students
author chosen for the taxi ride
12
10
8
16 Italian
6
12 non Italian
4
2
0
Feyerabend Levi Strauss
Semprini
anybody
28 Italian and non Italian students
comprehensibility, interest and emotion
30
25
20
Feyerabend
15
LeviStrauss
10
Semprini
5
0
more
comprehensible
more interesting more emotional
In deep-interviews with students
1.
2.
3.
4.
Authors
Texts
Reading act
Context
Students‘ metaphors
• The invisible author: realistic, precise,
scolastic, serious, authoritative, rigorous,
clear, scientific, vague, hostile, sterile, as
an anchor, it does not touch
• Reading the invisible author’s texts:
heavy, dense, rigid, tiring, hard, twisted,
muddled, cold, abstract, complex, distant
Students‘ metaphors
• The audio-visible authors: sense of movement,
participation, feeling of nearness and familiarity,
involvement, relationship, access, contact, possibility of
identification with the author, clearity, fresh air, sympathy,
an encountering that leaves a trace, convincing effect,
influencing effect, openness, sense of freedom, an
embrace, a testimony, “having more than a weapon (to
study)” (italian expression), a breathing, an encountering
that touches
• Reading the audio-visible authors’s texts: light, fluid,
a flowing and pleasant experience, linear, untied, warm,
concrete, easy, near
3.rd case study
M. Foucault – audio-visible author
– (difficult to read and study)
B. Constant – invisible (only text) but visible
(I-you) author
– (easy to read and study)
grades
7.8
7.65
7.62
7.6
7.4
7.2
7.18
mean grades Ita
7
6.8
6.8
6.6
6.4
6.2
Foucault
Constant
mean grades non Ita
words
250
201.5
200
177
152.5
150
124
mean words Ita
mean words non Ita
100
50
0
Foucault
Constant
quotes
3
2.49
2.5
2.42
2
means quotes Ita
1.36
1.5
1.05
1
0.5
0
Foucault
Constant
means quotes non Ita
Additional audio-visible authors?
would you have preferred to have other authors
explained in this course in an audio-visible
format?
60
students
50
40
YES
30
INDIFFEERENT
20
NO
10
0
1
Some conclusions
• 2 language and multimedia
– (Inglese, Mayer, Rigotti, 2007, in press)
• Literacy and digital media
• Importance of author‘s awareness and the social
presence of the author in expository texts
• Designing instructional messages more speechlike
• Reconsidering Orality in combination with
Writing
• Importance of using archives for educational
purposes, for example TV and radio ones
Some interpretations
• For Brandt (1990), reading is considered as an
involvement act, where readers try to reach
across texts to other human beings, having to
be more consciously aware of what is taking
place on the other side of the communication, as
is necessary when the discourse is oral.
• Readers read not to separate from others,
but to reach out to them. The motive for
reading is to find other minds.
Some interpretations
• Literacy failures are not failures of
separation but rather failures of involvement.
They arise not from overdependence on context
but from the lack of access to a context for
making sense of print.
• Instead of viewing the oral as antagonist to
the literature, it is necessary to understand
better how the oral sustains the literate.
• (Brandt, 1990, p. 7)
Looking for UCSB collaboration
• accessing US grant money and US
scholars that would like to use these
audio-visible authors too, for instructional
purposes and for research!
• developing together the Social Science
audio-visible authors‘ database in
collaboration with the RTSI Swiss public
TV and USI Lugano.