Sue Nichols
Centre for Research in
Education
Multimodality
The term ‘multimodal’ is used to refer to a text
constructed from different kinds of meaningmaking resources eg print language, sound &
visual images.
Some resources used by text producers:
Multimodality
The term ‘multimodal’ is used to refer to a text
constructed from different kinds of meaningmaking resources eg print language, sound &
visual images.
Some resources used by text producers:
Graphic symbols
Spoken language
Sound effects
Moving images
Hyper-linking
Concrete materials
Written language
Sally (aged 4) uses visual resources
CONTEXT: Sally had noticed that a
researcher visiting her
kindergarten was writing in a
notebook. She asked to draw
something in the notebook. A
yoghurt container full of coloured
feathers was on the table. Sally
announced she would draw that
and asked if the researcher would
draw the container first. Then the
following conversation happened
…
Sally uses spoken language and social
resources
Sally: Sally has green feathers
and red feathers and purple
feathers and yellow
feathers and purple flowers.
You can write it in your
story now.
Sally: You do it like that. These
are the fluffy bits. You have
to go down to here. You
have to be careful with
textas. You’re doing it nicely,
aren’t you?
Sue: OK.
Sue: How’s that?
Sally: I’m going to draw a
feather. I’ll do the purple
one. You do the next one.
Sally: Good. I’ll do the next one.
Sue: OK. I’ll do the green one.
Sally employs written language resources via an
expert helper.
Here, the adult helper is the resource, or indeed the
technology, which is utilised by Sally to produce the
written part of her text.
Multimodality is not new
• However, digital technologies offer a
different set of resources for producing
and consuming multimodal texts.
• They can make it easier to work across
modes and to include modes that have
become neglected.
Digital word processing has been changing the nature
and pace of children’s writing development
One of the children had actually
sent e-mail messages in September
with spaces between words, but
did not begin to consistently add
spaces in her handwriting for 18
months.
Yost, N. (2003) Computers, Kids, and Crayons: A comparative study of emergent
writing behaviours. IFIP Working Group 3.5 Conference: Young Children and
Learning Technologies, held at UWS Parramatta
One child typed his name,
“Jeffery”, highlighted it, selected
the WingDing font type, and
gleefully exclaimed, “I made my
name in code.” The typed name
“Jeffery” had become
jefferry
Digital technologies can create different
opportunities for children’s communication.
•
•
•
•
New audiences
New forms of written communication
New kinds of collaboration
Students becoming ‘prosumers’
Collaborative online writing
In the United States, Common
Core State Standards (CCSS)
requires teachers to “engage
students in collaboration and
online publication of writing”.
Teachers Mary Blow and
Ronica Lawrence, working in
schools over 100k apart,
decided to partner their classes
in an online science research
project.
http://www.scholastic.com/teachers/top-teaching/2012/01/collaborative-writing-and-online-publishing accessed 22/8/12
Online collaboration using Knowledge Forum
Children could see the
photographs and the ideas
posted by their peers. By
clicking on a peer’s entry, text
scaffolds in the database
would appear that gave
children choices; for example:
“my theory is…” or “I want to
know.”
Many children were motivated
to make entries themselves
using whatever strategies they
had available, including use of
invented spelling. Pelletier et al 2012
Integrating reading & writing with
digital devices
Amy ( aged 7) & Winnie (aged 8) made notes on their Kindles while reading
Friendship According to Humphrey .
Amy made 43 notes and Winnie made 33.
don’t worry Humphrey i
have a terrible life with
my sister
that is werd
what does that line mene
i wonder what garths
house looked like
Sourced from Larson, L. (2010) Digital Readers: The next chapter in e-book reading and response. The Reading Teacher
64(1) pp 15 – 22.
Literacy Apps – Blending old and new
www.curiouspuppy.com
This app is a version of a class children’s book.
It is not unusual to see older children’s
literature appearing in app form, as it is
cheaper for producers than employing authors
to create original content. Given what Susan
Hill’s research is revealing about the
popularity of mobile phones with very young
children, this offers a way to engage them
with quality children’s literature at little cost
to parents.
First Published 1979
• What has changed?
• How many of these functions could be carried out without
digital technology?
• There are significant continuities in so-called ‘New Literacy’
texts and practices.
Personal Examples
• E-book making in libraries
• Photo studio in classrooms
In the following slides I share some examples of multimodal texts
produced by children in two projects I have been involved in.
E-book Making in Libraries
• Sessions for children and
carers were run in school
holiday programs at two
metropolitan libraries.
PRE-PRODUCTION:
1.
2.
• Separate sessions were run for
younger (aged 3 – 6) and older
(aged 7 – 10) children.
3.
• Sessions were 90 minutes long
and utilised the iPad app ‘My
Story’.
• Sessions began with a ‘hands
off’ pre-production phase.
Icon recognition (It is vital for
users to understand how to
access functions like ‘undo.’)
Explanation of multiple modes &
non-linear production (Users can
commence creating their story
with drawing, photography,
writing or oral narration.)
Narrative planning ( A ‘hands-off’
time for children and their carers
to decide on characters and story
line. Deciding on a title is a good
focusing move.)
PRODUCING THE E-BOOK
RESOURCES:
• iPads
• paper & textas
• puppets & figurines
• large pieces of
coloured cardboard
as backdrops
• cubes for setting up
scenes
PROCESS:
• Most children began with the
visual element. Children chose to
either draw or to stage and
photograph scenes using props.
Some combined both.
• The language part of the
narrative was either dictated to
an adult to type or was recorded
as an oral narration by the child.
Some children chose to type their
own text.
• Narratives that had been well
planned could be produced more
efficiently since children generally
moved quickly to the
complication part of the story.
NARRATED BY CHILD: Once there was a princess
NARRATED BY CHILD: Once the princess met a dragon.
NARRATED BY CHILD: The dragon ate the princess.
NARRATED BY CHILD: He didn’t like her and he spat her out.
Created on an iPad by ‘Jack’, a Junior Primary child, using
photography, drawing, handwritten text & text typed by an
adult but no oral narration.
Once there was a fire breathing dragon
One day the dragon found that he
couldn’t breath fire any more.
So the dragon went to see the giraffe.
The giraffe waved his magic wand.
Suddenly the dragon could breathe fire
again. The end.
Classroom Photo Studio: Year 1 Children making
design choices
• For this project, children worked
with digital cameras and
PowerPoint to produce
representations of significant
personal objects.
• A photo booth was set up in the
classroom with a digital camera,
tripod, table and display board.
Cardboard provided the
backdrop.
PROCESS:
• Children posed and
photographed their objects.
• The photographs were saved to
the class computer. Children then
took turns to dictate written text
in one or more of three formats:
– Speech bubble
– Thought bubble
– Caption
•
All the slides were combined in
a class presentation that was
shown during the school’s Art
Night.
Eagles are
scary.
DESIGN DECISIONS:
What object to show;
How to pose it;
Whether to add text;
What text to add;
Speech or thought bubble or
caption;
Colour for text and fill.
Wheeeeee!!!
My car knows everything about the world. My
car has been all over the world
Images of his toy shark taken by a 5 year old boy
Adult: Is this how you want it to
look?
Child: Shakes head
Adult: How do you want it to look?
Child: Fierce.
Image selected by child.
Propping shark on plastic cup to show mouth.
Questions about any technology
•
•
•
•
Is it flexible?
Can it be used in combination with existing tools?
Does it foster the integration of skills?
Does it facilitate both scaffolded and
independent use by children?
• Can it be used for text production as well as for
the consumption of texts produced by others?
• Does it support different ways of learning and of
displaying learning?
REFERENCES
•
•
•
•
Larson, L. (2010) Digital Readers: The next chapter in e-book reading and response.
The Reading Teacher 64(1) pp 15 – 22.
Pelletier, J., Reeve, R., & Halewood, C. (2006). Young children's knowledge building
and literacy development through Knowledge Forum. Early Education and
Development, 17(3), 323 - 346.
Yost, N. (2003) Computers, Kids, and Crayons: A comparative study of emergent
writing behaviours. IFIP Working Group 3.5 Conference: Young Children and
Learning Technologies, held at UWS Parramatta
http://www.scholastic.com/teachers/top-teaching/2012/01/collaborative-writingand-online-publishing accessed 22/8/12
TO REFERENCE THIS PRESENTATION:
Nichols, S. (2013) Multimodal literacy learning with interactive technologies.
Presentation for the Centre for Research in Education, University of South Australia,
Adelaide, July 3rd 2013.