THE USE OF GESTURES IN LANGUAGE ANIMATION STUDY
SUBMISSION TYPE: POSTER PRESENTATION ONLY
ABSTRACT:
In line with embodied accounts of cognition, multiple studies have suggested a link between
semantics and the motor system of the human body (e.g. Engelkamp & Zimmer, 1997; Zwaan et
al., 2010). In educational settings, it has been shown that children, when instructed to gesture
(i.e. using the motor system) while explaining math problems, added new problem-solving
strategies to their repertoire and remembered more from a subsequent lesson (Broaders et al.,
2007). Considering language acquisition, gesture research has focused on second language
learning and on concrete topics (e.g. word learning; Tellier, 2008). However, little research has
focused on first language acquisition and more abstract concepts (e.g. grammar rules). Moreover,
there is little research that has investigated the use of gesturing in the context of learning from
instructional animations (De Koning & Tabbers, 2011). In the present study, we examined
whether the use of gestures during animation study facilitates the acquisition of certain
grammatical rules. Sixty-seven children (10-13 years old) watched animations in which an active
sentence, such as ‘Kim is reading the book’ was transformed into a passive sentence (‘The book
is being read by Kim’). For half of the children, a human hand appeared on the screen, moving
the words to the right places. These children were instructed to gesture along, as if they were
moving the words themselves. The other half of the children did not see the hand and was not
instructed to gesture along. They were tested on acquisition of the grammatical rule immediately
and one week after the study phase. We hypothesized that children in the gesture condition
would perform better on both posttests. Against expectations, children in the gesture condition
performed worse than children in the no gesture condition. Separate analyses of children with
high and low levels of language skills revealed that the negative effect of gesturing was only
present for the children with low language skills. Also, these children reported a higher amount
of mental effort invested in understanding the animations than children with high language skills.
It seems that, especially for children with low language skills, the extra instruction to gesture
resulted in some kind of overload instead of facilitation of acquisition. We discuss the results in
light of embodied cognition and instructional design. We propose several ideas about how
gestures might facilitate grammar learning from animations.
References
Broaders, S.C., Cook, S.W., Mitchell, Z., & Goldin-Meadow, S. (2007). Making children gesture
brings out implicit knowledge and leads to learning. Journal of Experimental Psychology:
General, 136, 539-550.
De Koning, B.B., & Tabbers, H.K. (2011). Facilitating understanding of movements in dynamic
visualizations: An embodied perspective. Educational Psychology Review, 23, 501-521.
Engelkamp, J., & Zimmer, H.D. (1997). Sensory factors in memory for subject-performed tasks.
Acta Psychologica, 96, 43-60.
Tellier, M. (2008). The effect of gestures on second language memorisation by young children.
Gesture, 8, 219-235.
Zwaan, R.A., Taylor, L.J., & de Boer, M. (2010). Motor resonance as a function of narrative
time: Further tests of the linguistic focus hypothesis. Brain & Language, 112, 143-149.