Contemporary Educational Psychology 39 (2014) 275–286 Contents lists available at ScienceDirect Contemporary Educational Psychology j o u r n a l h o m e p a g e : w w w. e l s e v i e r. c o m / l o c a t e / c e d p s y c h Theoretical Analysis Drawing pictures during learning from scientific text: testing the generative drawing effect and the prognostic drawing effect Annett Schmeck a,*, Richard E. Mayer b, Maria Opfermann a, Vanessa Pfeiffer c, Detlev Leutner a a Department of Instructional Psychology, University of Duisburg-Essen, Berliner Platz 6-8, D-45127 Essen, Germany Psychology, University of California at Santa Barbara, Santa Barbara, CA 93106, USA c Didactics of Biology, University of Duisburg-Essen, Universitätstraße 2, D-45117 Essen, Germany b A R T I C L E I N F O Article history: Available online 23 July 2014 Keywords: Text comprehension Drawing Generative learning activities Generative drawing effect Prognostic drawing effect A B S T R A C T Does using a learner-generated drawing strategy (i.e., drawing pictures during reading) foster students’ engagement in generative learning during reading? In two experiments, 8th-grade students (Exp. 1: N = 48; Exp. 2: N = 164) read a scientific text explaining the biological process of influenza and then took two learning outcome tests. In Experiment 1, students who were asked to draw pictures during reading (learnergenerated drawing group), scored higher than students who only read (control group) on a multiplechoice comprehension test (d = 0.85) and on a drawing test (d = 1.15). In Experiment 2, students in the learner-generated drawing group scored significantly higher than the control group on both a multiplechoice comprehension test (d = 0.52) and on a drawing test (d = 1.89), but students who received authorgenerated pictures in addition to drawing or author-generated pictures only did not. Additionally, the drawing-accuracy scores during reading correlated with comprehension test scores (r = .623, r = .470) and drawing scores (r = .620, r = .615) in each experiment, respectively. These results provide further evidence for the generative drawing effect and the prognostic drawing effect, thereby confirming the benefits of the learner-generated drawing strategy. © 2014 Elsevier Inc. All rights reserved. 1. Introduction Suppose you want to enable students to study a scientific text by themselves for deep level understanding. In this case, you will have to ensure that students engage in generative learning processes during reading, such as organizing material into coherent mental representations, and integrating the representations with each other and with relevant knowledge activated from long-term memory (de Jong, 2005; Mayer, 2004, 2009; Wittrock, 1990). A possible way to accomplish this goal is to encourage students to use a learner-generated drawing strategy (Alesandrini, 1984; Schwamborn, Mayer, Thillmann, Leopold, & Leutner, 2010; van Meter & Garner, 2005), in which they receive a text to read and are instructed to draw pictures that reflect the main elements and relations described in the text. The goal of the present study is to examine a generative drawing effect (i.e., engaging in appropriate drawing activities during learning from text improves performance on tests of learning * Corresponding author. Address: Annett Schmeck, University of Duisburg-Essen, Berliner Platz 6-8, D-45127 Essen, Germany. Fax: +49 2011834350. E-mail address: annett.schmeck@uni-due.de (A. Schmeck). http://dx.doi.org/10.1016/j.cedpsych.2014.07.003 0361-476X/© 2014 Elsevier Inc. All rights reserved. outcomes) and a prognostic drawing effect (i.e., the quality of drawing during learning from text predicts performance on subsequent tests of learning outcomes). 1.1. Theoretical framework for the learner-generated drawing strategy A straightforward way to encourage students to use a learnergenerated drawing strategy when learning from verbal instruction is to ask them to generate an external visual representation of a tobe learned content. The drawing that is generated has a representational quality, similar to the characteristics of a representational illustration (cf., Alesandrini, 1984; van Meter & Garner, 2005). By representational, we mean that learners make drawings which are intended to show what depicted objects look like (Carney & Levin, 2002). This requirement excludes nonrepresentational graphic constructions such as diagrams and concept maps. Thus, our definition of drawing is that the learner creates a visual representation intended to depict what is described in text. Drawing can be seen as a learning strategy intended to influence how learners process information during learning (Pashler et al., 2007; Weinstein & Mayer, 1986). By drawing, learners are no longer passive consumers of information and knowledge; they are 276 A. Schmeck et al./Contemporary Educational Psychology 39 (2014) 275–286 actively involved in the cognitive processes of selecting, organizing and integrating the information to be learned. Thus, learnergenerated drawing is a cognitive learning strategy that is aimed to foster learning from text, and if used adequately drawing can increase learning outcomes (Ainsworth, Prain, & Tytler, 2011; Alesandrini, 1984; van Meter & Garner, 2005). The processes underlying drawing are described in van Meter and Garner’s (2005) generative theory of drawing construction (GTDC), which is based on Mayer’s (2005) model of multimedia learning. It is assumed that learners benefit from using the drawing strategy as drawing requires them to engage in generative learning processes during reading. First, learners select the relevant key information from the text. Second, the selected key information is organized to build up an internal verbal representation of the text information. Third, learners construct an internal nonverbal (visual) representation of the text information and connect it with the verbal representation and with relevant prior knowledge. To construct the visual representation, which is the basis for the external drawing, the learner has to rely mainly on the verbal representation, and thus learner-generated drawing demands an integration of the verbal and nonverbal representation. Additionally, van Meter and Garner (2005) describe metacognitive processes fostered by the drawing activity: “Attempts at constructing the nonverbal representation can send learners back to either the verbal representation or the text as difficulties building the internal image are encountered” (van Meter & Garner, 2005, p. 317). That is, as the drawing process itself is not linear, metacognitive processes of monitoring and regulation are stimulated by drawing (cf., van Meter, 2001; van Meter, Aleksic, Schwartz, & Garner, 2006). 1.2. Empirical framework for the learner-generated drawing strategy Following the GTDC (cf., van Meter & Garner, 2005), the drawing strategy is beneficial as it fosters deep cognitive processing including organizing and integrating material (which can be called generative processing; Mayer, 2009) as well as metacognitive selfmonitoring and regulation processes. Research on drawing, however, has produced somewhat mixed results (see Alesandrini, 1984; van Meter & Garner, 2005, for overviews) in which some studies reported positive effects of drawing on text comprehension (e.g., Alesandrini, 1981; Hall, Bailey, & Tillman, 1997; Leopold & Leutner, 2012; Lesgold, DeGood, & Levin, 1977; Lesgold, Levin, Shimron, & Guttman, 1975; Schwamborn et al., 2010; van Meter, 2001; van Meter et al., 2006), whereas others did not (e.g., Leutner, Leopold, & Sumfleth, 2009; Rasco, Tennyson, & Boutwell, 1975; Tirre, Manelis, & Leicht, 1979). Benefits of drawing appear to be related to the quality of students’ drawings during learning: Students, who produce high-quality drawings during reading, tend to score better on posttests of learning outcome than do students who produce lowquality drawings during reading (e.g., Greene, 1989; Hall et al., 1997; Leopold, 2009; Lesgold et al., 1975, 1977; Schwamborn et al., 2010; van Meter, 2001; van Meter et al., 2006). 1.2.1. Effectiveness of learner-generated drawings Following van Meter and Garner (2005), reasons for the mixed empirical results concerning drawing can be seen attributed to the type of test used for assessing learning outcomes as well as in the form of support that assists learners in the drawing process. First, benefits of drawing are more likely to be revealed on tests that assess higher-order knowledge of to-be learned content, for example, tests on comprehension and transfer (e.g., Alesandrini, 1981; Leopold & Leutner, 2012) or problem solving (van Meter, 2001; van Meter et al., 2006). Leutner et al. (2009), for example, found no positive effect of drawing compared with a control group on a multiple choice test on factual knowledge. Leopold and Leutner (2012), however, showed superior effects of the drawing strategy on transfer test perfor- mance. van Meter et al. (2006), accordingly, found no effects of drawing activity on a multiple choice recognition test; however, students in the drawing group scored significantly higher on a problemsolving test. With regard to the GTDC (van Meter & Garner, 2005), it seems that benefits of drawing can be found if the learning outcome test complies with characteristics of the verbal and nonverbal representations, which are generated by drawing. Second, positive effects of drawing often appear under the condition that instructional support is provided to constrain and structure the drawing activity (e.g., Lesgold et al., 1975, 1977; Schwamborn et al., 2010; van Meter, 2001; van Meter et al., 2006). That is, drawing is more effective when the learners’ generation of the drawing is assisted by some kind of additional information. van Meter (2001) and van Meter et al. (2006), for example, showed that the provision of author-generated pictures after drawing enhanced the benefits of the drawing strategy. By comparing their own drawing with a provided one, learners get to know what their drawing should look like, and this might lead them back to revise their own drawing and thus, their mental model. Following the GTDC (van Meter & Garner, 2005), this should improve comprehension. Lesgold et al. (1975), in turn, supported first grade students with cutout figures and instructed them to organize these into an accurate pictorial representation while listening to a prose story. This learner-generated illustration activity facilitated prose learning as indicated by higher recall of story propositions only when students were given the correct pieces for the illustration or had the illustration done for them. When students had to select the pieces for each illustration out of a pool of cutouts, the learner-generated illustration activity had either a negative or no effect (cf., Lesgold et al., 1977). Following these results, Schwamborn et al. (2010) proposed that a pure, unsupported drawing instruction might bear the risk that managing the mechanics of drawing itself is difficult for the learners, resulting in insufficient remaining capacity for making sense of the text through generative processes of organization and integration, which might diminish the benefits of drawing defined by van Meter and Garner’s GTDC. To counter this risk in the study of Schwamborn et al. (2010), students in the drawing groups received baseline instructional support while learning a lesson on washing, which provided them with a drawing prompt that included a legend showing all the relevant elements for drawing and a partly pre-drawn background for their paper-pencil based drawings. That is, students could use the presented elements as prototypes for their own drawings and integrate them by pencil in the given pre-drawn backgrounds. Results showed that students, who were instructed to generate drawings during learning, scored significantly higher on the subsequent comprehension tests than students who only read the text. Using cutout-figures (cf., Lesgold et al., 1975, 1977) or a drawing prompt (cf., Schwamborn et al., 2010) during drawing seems to provide sufficient constraints and leave enough cognitive capacities for learners to benefit from the drawing strategy. Thus, cognitive processing including selecting, organizing and integrating material should be encouraged, resulting in an improved mental model, which in turn should improve comprehension (cf., GTDC; van Meter & Garner, 2005). In line with the GTDC and the reported results derived from research on drawing Schwamborn et al. (2010) proposed a generative drawing effect, that is, students gain a better understanding of a scientific text when they are asked to draw illustrations representing the content of each paragraph they read. This work highlights the importance of drawing support, such as the provision of drawings of all key elements and a background for the drawing. 1.2.2. Quality of learner-generated drawings Previous studies that measured the quality of students’ drawings during learning all showed positive correlations between A. Schmeck et al./Contemporary Educational Psychology 39 (2014) 275–286 the quality of students’ drawings during learning and their learning outcomes (e.g., Greene, 1989; Hall et al., 1997; Lesgold et al., 1975, 1977; Schwamborn et al., 2010; van Meter, 2001; van Meter et al., 2006). The quality of learner-generated drawing is also referred to as the drawing accuracy (e.g., van Meter, 2001; van Meter et al., 2006), and is defined as “the degree to which completed drawings resemble the represented object(s)” (van Meter & Garner, 2005, p. 299). In a study by Hall et al. (1997), for example, college students learning a mechanics lesson with the instruction to draw produced better learning outcomes on a transfer test than a text only control group, but only if they produced higher quality drawings. In the study by Schwamborn et al. (2010), students who were able to generate high accuracy drawings scored significantly higher on learning outcome tests than did those who generated lower accuracy drawings. In addition to this, Schwamborn and colleagues also found that the quality of the generated drawings during learning correlated positively with the comprehension test scores. Based on these results Schwamborn et al. (2010) proposed the prognostic drawing effect: Students, who produce high-quality drawings during reading a scientific text, tend to score better on posttests of learning outcome than do students who produce low-quality drawings during reading. 1.3. Overview of the experiments Research on drawing – i.e., the generative and the prognostic drawing effect – is promising; however, at the present time there is a need for a more solid evidence base and for a closer examination of theoretical issues. First, the generalizability is limited at this point as replication studies using learning outcome tests that are sensitive to the underlying process of drawing as well as new learning materials other than the washing lesson (e.g., Schwamborn et al., 2010) or the birds wing (van Meter, 2001) are yet missing. In their report for the U.S. National Research Council, entitled Scientific Research in Education, Shavelson and Towne (2002, p. 4), for example, highlighted the need to “replicate and generalize across studies” as one of the six essential scientific principles of educational research. It has to be mentioned at this point, that when generalizing results to new domains or lessons, one should carefully consider whether these are comparable at all. In our experiments, we aimed at generalizing results by Schwamborn et al. (2010), who worked with a science text explaining the causal steps regarding the chemistry of washing, to a new lesson that is, however, comparable in that the text we used also described causal steps of a process, in this case regarding the infection with influenza and regarding the immune response. That is, although there were differences between the two domains (chemistry versus biology), the lessons showed structural similarities and thus allow for comparing results and drawing conclusions regarding generalizability. Second, research on drawing indicates that some form of support is needed to assist learners during drawing. Schwamborn et al. (2010), for example, introduced a drawing prompt as helpful support for learners to benefit from drawing. They proposed that the resulting positive drawing effect is based on students’ engagement in generative learning activities during reading due to drawing (consistent with the GTDC of van Meter & Garner, 2005; see also de Jong, 2005; Mayer, 2004, 2009; Wittrock, 1990). However, the results reported by Schwamborn et al. (2010) might rather reflect a multimedia effect (Mayer, 2005, 2009) than the proposed drawing effect as the learning lesson used – a scientific text and a drawing prompt consisting of pictorial elements and backgrounds – created a multimedia lesson. In other words, the results of Schwamborn et al. (2010) might not be due to the drawing activity but rather due to the multimedia effect that students “learn better from words and pictures than from words alone” (Mayer, 2009, p. 223). In this case, 277 the words are presented in the lessons and pictures are generated by the students, so a control group that receives author-generated pictures is warranted. Third, research on drawing mostly used only one way to support the drawing strategy at a specific time. That is, instructional support was added during learning (i.e., by using cut-out figures or a drawing prompt; (cf., Lesgold et al., 1975, 1977; Schwamborn et al., 2010) or after learning (i.e., by providing pictures, van Meter, 2001; van Meter et al., 2006). Less is known about whether adding instructional support not only during learning but also after learning can additionally enhance the benefits of the drawing strategy. Fourth, research on drawing should include motivational and cognitive aspects that may have an impact on the effectiveness of the learner-generated drawing strategy. Students’ current motivation, for example, is a one condition for successful learning. A student, for example, who has low motivation to learn, may invest less effort in learning than students who are highly motivated to learn (cf., Vollmeyer & Rheinberg, 2000). Students’ spatial ability may be a further condition for successful learning when working with visualizations (cf., Höffler, 2010; Höffler, Schmeck, & Opfermann, 2013). A high-spatial-ability student, for example, may have advantages in learning with visualizations compared with a lowspatial-ability student. That is, preexisting motivational and cognitive differences between students before learning might have an influence on the learning outcome, and thus should be controlled. In addition, recent research has shown that not only experimental conditions (such as the kind of picture) and the above mentioned “classical” covariates can have an impact on how successful learning takes place, but that these effects can be mediated by the amount of mental effort someone invests while learning or working on a lesson; and by how difficult someone perceives a domain or lesson to be (cf., Leutner et al., 2009; Schwamborn, Thillmann, Opfermann, & Leutner, 2011). These aspects of cognitive load (invested mental effort and perceived task difficulty) were thus included as additional variables in our studies as well. Thus, we conducted the following two experiments using a science text explaining the biological process of influenza. In Experiment 1 we implemented an experimental drawing condition and a reading only control condition, in order to determine how both the generative and the prognostic drawing effect would extend to a new context. Analogous to the study of Schwamborn et al. (2010), students in the drawing condition received a baseline instructional support by means of a drawing prompt that included a legend showing all the relevant elements for drawing and a partly pre-drawn background for their drawing (as shown in Fig. 1). In Experiment 2, we again implemented an experimental drawing condition and a reading only control condition, and we additionally implemented author-generated pictures, in order to test whether the generative drawing effect was caused by the simple presence of illustrations rather than the generation of illustrations. That is, we implemented a text plus picture condition (which we called the author-generated picture condition), to test whether the reported generative drawing effect of Schwamborn et al. (2010) is based on students’ engagement in generative learning activities during reading rather than on the pictorial representations given by the drawing prompt. In addition, we implemented a drawing plus picture condition (in which students both draw and are given a picture), to test whether the reported generative drawing effect can be enhanced by instructing students to compare their own drawing with an author-generated picture. In short, we tested whether combining different forms of support to the drawing strategy additionally enhances the benefits of the drawing strategy. In both experiments, learning outcome tests that are sensitive to the underlying process of drawing were used. 278 A. Schmeck et al./Contemporary Educational Psychology 39 (2014) 275–286 Fig. 1. Screenshot of the drawing prompt for the first paragraph in the drawing versions of the learning booklet. Note: Translated from the German original. 2. Experiment 1 four students served in the control group, and 24 served in the drawing group. 2.1. Participants and design 2.2. Materials Forty-eight German eighth graders in higher track secondary schools participated in this study. The mean age was 13.7 years (SD = 0.6), and there were 22 girls and 26 boys. The study was based on a between-subjects design, with two levels of text learning (learner-generated drawings and control) as the single factor. Twenty- All materials were paper-pencil based. The materials consisted of five adjunct questionnaires, two learning booklets, two cognitive load rating scales, and two posttests. The five adjunct questionnaires were intended to determine whether the groups were A. Schmeck et al./Contemporary Educational Psychology 39 (2014) 275–286 equivalent on basic characteristics. They included a participant questionnaire, a comprehension pretest, a spatial ability test, and a motivation questionnaire. The participant questionnaire solicited information concerning the students’ age and sex. The comprehension pretest consisted of 25 multiple-choice items and was intended to assess students’ prior-knowledge of information covered in the text. Students’ spatial ability was measured with a 10 multiple-choice paper-folding items taken from a battery of cognitive tests developed by Ekstrom, French, and Harman (1976). The motivation questionnaire assessed students’ current motivation for doing the learning task after reading the instructions before the lesson. It consisted of nine items from the challenge and interest subscales of the Questionnaire on Current Motivation (QCM) developed by Rheinberg, Vollmeyer, and Burns (2001). Cognitive load by means of invested mental effort was measured using the 7-point subjective rating scale developed by Paas (1992), which ranges from (1) very low mental effort to (7) very high mental effort. Cognitive load by means of perceived task difficulty was measured using the 7-point subjective rating scale developed by Kalyuga, Chandler, and Sweller (1999), which ranges from (1) very easy to (7) very difficult. These subjective measures have been criticized for assessing cognitive load with only single items (e.g., Brünken, Plass, & Leutner, 2003). However, several studies showed the effectiveness of the rating scale by showing that the variation in learners’ cognitive load ratings depended on variations in task complexity or instructional design (for overviews see Paas, Tuovinen, Tabbers, & Van Gerven, 2003; Van Gog & Paas, 2008). In this regard, Sweller, Ayres, and Kalyuga (2011) conclude that “the simple subjective rating scale, regardless of the wording used (mental effort or difficulty), has, perhaps surprisingly, been shown to be the most sensitive measure available to differentiate the cognitive load imposed by different instructional procedures” (p. 74). For that reason and due to the economic applicability we decided to use this kind of cognitive load measurement, while acknowledging the limitations of a short, self-report instrument. The two learning booklets each included a science text on the biology of the influenza. The text explained the causal steps regarding an infection with influenza and regarding the immune response, which is an unfamiliar subject for eighth graders in higher track secondary schools due to the German curriculum. The text consisted of approximately 850 words (in German) and was divided into seven paragraphs (as shown in Table 1). The drawing version of the booklet contained seven pairs of facing pages with a text paragraph on the left page and a two-part drawing prompt on the right page. The first part of the drawing prompt included a legend showing all the relevant elements (in total eight elements) for drawing a picture for that text paragraph (as shown in the top of Fig. 1). The second part of the drawing prompt included a partly pre-drawn background for students’ drawing (as shown in the bottom of Fig. 1). Overall, students had to make seven drawings, i.e., one drawing to each paragraph. The control version of the learning booklet contained four pairs of facing pages with one of the seven text paragraph on each page. Table 1 Text from the second paragraph of the influenza lesson. How the influenza virus replicates Once inside the influenza virus uses your somatic cell to produce new particles of the influenza-virus. The glycoproteins move toward the membrane of the somatic cell and stick out into the outside of the cell. The capsules of the virus, however, are assembled inside the somatic cell. Next, these new assembled capsules of the virus leave your somatic cell. By moving through the somatic cell membrane the capsules are enveloped with the membrane and its glycoproteins, which then plays the role of the virus membrane. Thus, several new influenza viruses are located outside your somatic cell. Note: Translated from the German original. 279 Students in both groups learned with exactly the same text material. To make sure that students in the control group learned with the same amount of information as students in the drawing group, all elements of the drawing prompt as well as the spatial relations between these elements were also described in the science text. The two posttests intended to assess the learning outcomes were a comprehension posttest and a drawing posttest. The comprehension posttest (Cronbach’s alpha = 0.83) consisted of 25 multiplechoice items (the same items as in the comprehension pretest) and was intended to assess students’ comprehension of the factual and conceptual information covered in the text as well as their ability to transfer what was presented to new situations. An item example is “T-helper cells do not only recognize viruses, but also agents that are extraneous to the body. Which medication would you administer to a patient, who has received a new kidney? (a) a medicine that suppresses the immune response of the body, (b) a medicine that activates the immune response of the body, (c) a medicine that contains antigens, or (d) a medicine that contains blood of the kidney donor” [(a) is the correct answer]. The drawing test (Cronbach’s alpha = 0.81) was intended to assess students’ comprehension of the conceptual information presented in the science text by means of drawing. That is, students had to reproduce the main ideas given in the text by drawing. It consisted of three drawing items, in which students were asked to draw sketches depicting key concepts of the text and their spatial relations. An item example for the drawing test is “How does an influenza virus invade a cell, and how is it reproduced?” The science text, the drawing prompt and the learning outcome tests were constructed by the first author in cooperation with a biology teacher. The materials were adapted from Schwamborn et al. (2010); however, using another science domain, and including measures of individual learning times and cognitive load. 2.3. Procedure Participants were tested in the schools’ classrooms. Within their classes, they were randomly assigned to one of the two groups. Groups were tested in separate classrooms, in order to insure that students in the drawing group did not feel rushed when students in the control group completed the task early. Each student was seated at an individual desk. First, students were given the participant questionnaire and the comprehension pretest to complete at their own rate. Second, students filled in the paper-folding test with a 3 min time limit. Third, students were given instructional booklets corresponding to their assigned group. After they had read the instructions for reading the booklets, students’ current motivation for doing the learning task was assessed. Next, students started learning with the text material corresponding to their treatment group. Students were instructed to carefully read the text on the biology of the influenza in order to comprehend the material. Students in the drawing condition were instructed to read the text and additionally to draw pictures for each text paragraph using the drawing prompt representing the main ideas of each text paragraph. That is, students had to use the pictorial elements given in the legend such as the virus as templates for their own paper-pencil based drawing across the pre-dawn background. Students in the control group were instructed to read the text for comprehension, but were not instructed to engage in drawing. Students in both groups learned at their own pace, whereby individual learning time was measured by the instructors in the classrooms. Fourth, in order to ensure comparable testing procedures after finishing learning with the whole learning material, students in both groups directly rated the amount of mental effort he or she had invested during learning and the amount of difficulty he or she had perceived during learning. Fifth, students received the comprehension posttest consisting of 280 A. Schmeck et al./Contemporary Educational Psychology 39 (2014) 275–286 the comprehension posttest and the drawing posttest. Students had 20 minutes time for completion, and did not have access to the science text or their drawings. Finally, students were thanked and debriefed. As students learned at their own rate, the whole procedure took about 70–90 minutes, depending on the individual testing times. 2.4. Results and discussion 2.4.1. Scoring The dependent variables were students’ scores on the comprehension and drawing posttests, students’ rating on the mental effort and the difficulty scales, and the drawing accuracy score indicating the quality of learner-generated drawings produced by students in the drawing group during the learning phase. The comprehension test score (pre- and posttest) for each student was computed by awarding 1 point for each correct answer, and by adding up the points to obtain the total comprehension score (out of a total possible of 25 points). Actual scores ranged from 3 to 24 points, with a mean of 13 points (SD = 5.3). Following Schwamborn et al. (2010), scoring of the drawing test was carried out by counting the total number of correct main ideas in each learner’s answer across the three drawing items. The main ideas were drawn out from both expert visualizations and a checklist specifying important relational features. Students could earn a maximum of 19 points on the drawing test. Two student assistants (teacher trainees in biology) scored the quality for each of the three drawings for each student with an acceptable inter-rater agreement of Goodman–Kruskal gamma of 0.90. Actual scores ranged from 0 to 18.5 points, with a mean of 7.7 points (SD = 4.7). Total scores of both the comprehension and the drawing test were transferred into z-standardized scores to make them comparable across studies. The drawing accuracy score (concerning drawing during learning in the drawing group) was computed by using a coding scheme adapted from Schwamborn et al. (2010), which was based on expert drawings and a checklist specifying important relational features of the drawings. Students could earn a maximum drawing-accuracy score of 22 points. Again the two student assistants scored each of the seven learner-generated drawings for each student with an acceptable interrater agreement of Goodman–Kruskal gamma of .92. Both coding schemes were constructed by the first author and a biology teacher. Actual scores ranged from 4 to 21 points, with a mean of 13.3 points (SD = 5.0). The total drawing accuracy score was again transferred into a z-standardized score. In addition the spatial ability test was scored by tallying the number correct out of 10, and the motivation questionnaire was scored by tallying the nine ratings on both subscales to a total score of motivation. Finally, for comparing performance across the different tests, the proportion correct on each test was computed by dividing the student’s obtained score by the total possible score. 2.4.2. Are the groups equivalent on basic characteristics? Before looking at treatment effects on the dependent variables, we analyzed whether the two groups differed on several control variables. A chi-square analysis indicated that there were no significant differences regarding gender (p = .562). Separate univariate analyses of variance (ANOVAs) revealed that the groups did not differ significantly on age, F < 1; on spatial ability, F < 1; or on motivation, F(1, 46) = 3.60; p = .064. However, groups differed significantly on prior knowledge, F(1, 46) = 38.90, p < .001, partial eta2 = .46, in that students in the drawing group scored significantly lower on the comprehension pretest (M = .10, SD = .15) than students in the control group (M = .34, SD = .12). Thus, we included students’ prior knowledge in the following analyses. Table 2 Mean proportion correct on the comprehension test and drawing test for two groups – Experiment 1. Group Type of test n Drawing Control 24 24 Comprehension test Drawing test M SD M SD .61* .44 .20 .20 .52* .28 .27 .11 Note: Asterisk (*) indicates significant difference from control group at p < .05. 2.4.3. Is there support for the generative drawing effect? Mean proportion correct and SDs on the comprehension and drawing posttests for both groups are presented in Table 2. Repeated measures univariate analyses of variance (ANOVA) with the comprehension pre- and post-test scores as the within-subject factors and group (drawing versus control) as the between-subject factor showed a main effect over time indicating that overall, participants reached significant knowledge gains between the comprehension pretest and the comprehension posttest, F(1, 46) = 98.97; p < .001; partial eta2 = .68. An interaction additionally showed that these knowledge gains were significantly higher for the drawing group than for the control group, F(1, 46) = 46.17; p < .001; partial eta2 = .50. For the drawing test, a repeated measures ANOVA was not possible, since these items were only used in the posttest. In this case, a univariate analysis of covariance (ANCOVA) predicting the drawing test score with group (drawing versus control) as the factorial independent variable and prior knowledge as a covariate showed that the drawing group scored significantly better than the control group on the drawing posttest, F(1, 45) = 13.49, p = .001, partial eta2 = .23.1 Cohen’s d favoring the drawing group over the control group was 0.85 for the comprehension posttest, and 1.15 for the drawing posttest, all of which are considered large effects. Thus, there is strong support for the generative drawing effect, as predicted. Additionally, results revealed that the drawing group needed significantly more learning time (M = 21.08 min., SD = 4.24) than the control group (M = 17.38 min., SD = 3.33), F(1, 46) = 11.34, p = .002, partial eta2 = .20. Thus, to test whether learning time mediates the positive effect of drawing on text comprehension, additional mediation analyses (Baron & Kenny, 1986) were calculated by including learning time as an additional predictor in the aforementioned linear model. A mediation effect would be detected if, in this case, effects of drawing on text comprehension would significantly decrease. Results of the mediation analyses showed that the effect of drawing on both comprehension test scores and drawing test scores was not fully mediated by learning time. That is, including learning time still revealed the interaction between group (drawing versus control) and time (pre- versus post) in that the drawing group had significantly higher knowledge gains than the control group on the comprehension test items (p < .001). Furthermore, the drawing group also still outperformed the control group on the drawing posttest after controlling for learning time (p = .009). Furthermore, results revealed that students in the drawing group rated their invested mental effort during learning significantly higher (M = 5.04, SD = 1.12) than students in the control group (M = 3.96, SD = 1.65), F(1, 46) = 7.05, p = .011, partial eta2 = .13. There was no difference between the two groups on the perceived difficulty item (drawing group: M = 4.08, SD = 1.50; control group: M = 4.25, SD = 1.22; F < 1). Thus, consistent with predictions concerning the generative drawing effect, there is partial support for the idea that 1 The exclusion of students’ prior knowledge in the reported analyses does not change the reported pattern of results. A. Schmeck et al./Contemporary Educational Psychology 39 (2014) 275–286 drawing causes students to engage in more generative processing during learning. Taken together, the results suggest that the drawing strategy encourages students to engage in generative processing during learning, as is indicated by their higher learning outcomes. Thus, the data provide further evidence for the generative drawing effect consistent with the results of Schwamborn et al. (2010). Additionally, results indicate that students in the drawing condition seem to invest more mental effort than students in the control group, without perceiving higher levels of difficulty. 2.4.4. Is there support for the prognostic drawing effect? Mean proportion correct on drawing accuracy during learning was .59 (SD = .23). A correlation analysis revealed that the drawingaccuracy score of learner-generated drawings correlated significantly with the comprehension posttest score, r = .620, p < .001, and with the drawing posttest score, r = .623, p < .001. Additional correlation analyses revealed that the drawing-accuracy score of learnergenerated drawings correlated significantly negatively with the perceived difficulty score, r = −.489, p = .015. There were no significant correlations between the drawing accuracy score and either the invested mental effort score, r = −.134, p = .533, the prior knowledge test score, r = −.004, p = .984, the spatial ability test score, r = .072, p = .739, or the motivation test score, r = .086, p = .690. Thus, as predicted, the data provide further evidence for the prognostic drawing effect consistent with the results of Schwamborn et al. (2010). In sum, the results of Experiment 1 are consistent with the prediction that students learn better from a science text when they are asked to draw illustrations representing the main ideas of the text and that the quality of the generated drawings during learning correlates positively with students’ text comprehension (e.g., Schwamborn et al., 2010; van Meter, 2001; van Meter et al., 2006). However, it might be argued that the reported results are due to the way we supported the strategy use. In other words the reported positive effect of the learner-generated drawing strategy might not be caused by students’ engagement in generative learning activities during reading (de Jong, 2005; Mayer, 2004, 2009; Wittrock, 1990) but rather by the additional pictorial information given in the drawing prompt. Additionally, looking at students’ learning outcomes, our results indeed show positive effects of drawing; however, mean scores of learning outcomes for the drawing group are medium-sized. Thus, it might be argued that the way we supported the strategy use was not fully sufficient. In other words, the reported positive effect of the learner-generated drawing strategy, i.e., the generative drawing effect, might be increased by giving students instructional support in addition to the drawing prompt (van Meter, 2001; van Meter et al., 2006; van Meter & Garner, 2005). To address these issues, we added two experimental conditions by implementing author-generated pictures in the design in Experiment 2. 3. Experiment 2 One possible issue with Experiment 1 is the type of control group used. In Experiment 1, following Schwamborn et al. (2010), we used a reading only control group, in which the control group learned with verbal information only. In the drawing group, however, students not only learned with verbal information but also with pictorial information given by the drawing prompt. Based on theories of multimedia learning the use of different forms of representations such as texts and pictures can promote learning in that “people learn better from words and pictures than from words alone” (i.e., multimedia principle; Mayer, 2009, p. 223) because in this case, both, a (verbal) propositional representation as well as a (pictorial) mental model are built up and are optimally integrated into one schema that can be stored in long-term memory (Schnotz, 2005). 281 This assumption is also in line with the dual-coding approach stated by Paivio (1986). In this regard, it might be argued that the reported drawing effect is actually a multimedia effect that is based on the presentation of text and picture rather than a generative drawing effect that is based on students’ active engagement in drawing activities during reading. In other words, instead of asking people to draw pictures representing the main ideas of the text, giving them text and author-generated pictures representing the main ideas of the text might be as good or even better. Thus, we included a condition in Experiment 2, in which we added authorgenerated pictures to the text. An additional issue with Experiment 1 is whether the reported generative drawing effect can be enhanced by using various forms of supporting the strategy. First, there is evidence that using a drawing prompt during learning seems to be effective in supporting the learner-generated drawing strategy by minimizing the creation of extraneous processing (cf., Schwamborn et al., 2010; see also Exp. 1). Second, research has shown that instructing students to compare their own drawing with an author-generated picture might be also effective in supporting the learner-generated drawing strategy as self-monitoring processes are enhanced (cf., van Meter, 2001). Up to now, however, there is no empirical evidence whether the combination of both ways to support the drawing strategy has an additive effect on learning outcomes. Thus, we included a further condition in Experiment 2, in which we combined both forms of strategy support. The main purpose of Experiment 2 was to test the generative drawing and prognostic drawing effects of learner-generated drawing as in Experiment 1, but, this time also compared with another control group (i.e., author-generated pictures). Additionally, we were interested in testing whether the benefits of the learner-generated drawing strategy can be increased when we instructionally support students not only with a drawing prompt but also with an authorgenerated picture after the drawing process. In this new treatment, we instructed students to draw a picture of the text content, and then to compare their own drawing with an expert picture. 3.1. Participants and design The participants were 168 German eighth graders from higher track secondary schools. The mean age was 13.8 years (SD = 0.6), and there were 112 girls and 56 boys. The study was based on a 2 × 2-between-subjects design, with learner-generated drawing (yes/ no) and author-generated picture (yes/no) as factors. Forty students served in the drawing group, 44 students served in the authorgenerated picture group, 41 students served in the drawing + authorgenerated picture group, and 43 students served in the control group. 3.2. Materials The materials were identical to those used in Experiment 1, except that we used a shortened version of the comprehension pretest that consisted of 19 rather than 25 items (Cronbach’s alpha = .70) and slightly extended versions of both the comprehension posttest (28 items, Cronbach’s alpha = .84) and the drawing test (four items with a maximum score of 21 points; Cronbach’s alpha = .78). The pretest was shortened, because the first experiment showed that the respective items were either much too easy or much too difficult and thus unsuitable to differentiate between successful and unsuccessful learners; thus we deleted these items in the second experiment. Furthermore, we decided to add some items to the comprehension posttest in the second experiment, because during data analysis of the first experiment, and after receiving some feedback from experts in the domain of biology, we recognized that a few items assessing transfer ability could be added. These transfer items, however, would have been unsuitable to be included in the pretest 282 A. Schmeck et al./Contemporary Educational Psychology 39 (2014) 275–286 Fig. 2. Author-generated pictures for the seven paragraphs in the author-generated picture versions of the learning booklet. Note: Pictures are scaled-down from the original format. because they are too difficult to answer without prior training in the topic. Additionally, author-generated pictures were used in the new conditions. The author-generated pictures were static functional pictures representing the main ideas of each paragraph and consisted of pictorial elements identical to those provided in the drawing prompt (as shown in Fig. 2). These pictures were constructed by the first author in cooperation with a biology teacher. The drawing version of the booklet was identical to that used in Experiment 1 (as shown in Fig. 1). The control version of the learning booklet was identical to that used in Experiment 1. The authorgenerated picture version of the booklet consisted of seven pairs of facing pages with a text paragraph on the left page and a corresponding author-generated picture (such as in Fig. 2) on the right page. The drawing + author-generated picture version of the booklet contained the material from the drawing version consisting of seven pairs of facing pages with a text paragraph on the left page and a two-part drawing prompt on the right page. In addition, attached to each page there was an additional page that students could fold out after having generated their drawing. When unfolding this additional page, a picture of that text paragraph right aside the drawing prompt was provided, and there was an additional instruction to compare the learner-generated drawing with the author-generated picture. Author-generated pictures were the same as in the authorgenerated picture version of the booklet. 3.3. Procedure The procedure was identical to that used in Experiment 1, except that there were two additional groups learning with authorgenerated pictures. Students in the author-generated picture condition were instructed to read the text and additionally to look at pictures representing the main ideas of each text paragraph. Students in the drawing + author-generated picture version of the booklet were instructed to read the text, to draw pictures for each text paragraph using the drawing prompt representing the main ideas of each text paragraph, and finally to compare their picture with an author-generated picture representing main ideas of each paragraph correctly. Students in all groups learned at their own pace, whereby individual learning time was measured by the instructors in the classrooms. Again, to ensure that students’ in both drawing groups did not feel rushed when students in the non-drawing A. Schmeck et al./Contemporary Educational Psychology 39 (2014) 275–286 group completed the task early, groups were tested in separate classrooms. 3.4. Results 3.4.1. Scoring All tests instruments were scored with the same procedures used in Experiment 1. Again two student assistants (teacher trainees in biology) scored each of the drawing test items and each of the seven learner-generated drawings for each student with acceptable interrater agreements (drawing test: Goodman–Kruskal gamma of .90; drawing-accuracy: Goodman–Kruskal gamma of .94). Actual scores ranged from 1 to 28 points (M = 15.3 points, SD = 5.8) for the comprehension test, from zero to 21 points (M = 10.9 points, SD = 5.3) for the drawing test, and from 2.75 to 21.5 points (M = 14.1 points, SD = 4.8) for drawing accuracy. Again, total scores of comprehension, drawing and accuracy were transferred into z-standardized scores. 3.4.2. Are the groups equivalent on basic characteristics? Before looking at treatment effects on the dependent variables, we analyzed whether the four experimental groups differed on several control variables. A chi-square analysis indicated that there were no significant differences regarding gender (p = .097). Separate univariate analyses of variance (ANOVAs) revealed that the groups also did not differ significantly on age, F < 1; on spatial ability, F(3, 164) = 1.20, p = .312; or on motivation, F(3, 164) = 1.22; p = .305. However, groups differed significantly on prior knowledge, F(3, 164) = 1.04; p = .010, partial eta2 = .07, in that students in the control group scored significantly higher on the comprehension pretest (M = .25, SD = .17) than students in both (p < .05) the authorgenerated picture group (M = .17, SD = .16) and the drawing + authorgenerated picture group (M = .15, SD = .12); the drawing group (M = .20, SD = .14) did not differ significantly from the other groups. Thus, we included students’ prior knowledge as a covariate in the following analyses. 3.4.3. Is there support for the generative drawing effect? A major goal in this experiment was to determine whether asking students to generate drawings to represent science text is a more effective learning strategy than asking students to learn with text alone or with text and author-generated pictures. In other words, we wanted to determine whether we could replicate and extend the learner-generated drawing effect. Additionally, we were interested in whether giving students an author-generated picture after drawing can increase the benefits of the learning strategy. Mean proportion correct and standard deviations on the comprehension and drawing tests for the four groups are presented in Table 3. The left portion of Table 3 summarizes the mean proportion correct on the comprehension test. A two-factorial analysis of covariance (ANCOVA) predicting learning outcomes (comprehension posttest score) with learner-generated drawing (yes/no) and Table 3 Mean proportion correct on the comprehension test and drawing test for the four groups – Experiment 2. Group Type of test n Learner-generated drawing Author-generated picture Learner-generated drawing + author-generated picture Control Comprehension test Drawing test M SD M SD 40 44 41 .63* .53 .51 .22 .19 .20 .66* .50 .63 .22 .25 .19 43 .52 .20 .30 .16 Note: Asterisk (*) indicates significant difference from control group at p < .05. 283 author-generated picture (yes/no) as the factorial independent variables and prior knowledge as a covariate showed a significant positive main effect of learner-generated drawing, F(1, 163) = 3.98, p = .048, partial eta 2 = .02, a significant interaction effect, F(1, 163) = 6.26, p = .013, partial eta2 = .04, but no main effect of authorgenerated pictures, F < 1. In addition, multiple pairwise comparisons (with p < .05) showed that the drawing group performed significantly better than each of the three other groups, which did not differ significantly from each other. Cohen’s d favoring the drawing group over the author-generated picture group was .49, over the learnergenerated + author-generated picture group was .57, and over the control group was .52. The right portion of Table 3 summarizes the mean proportion correct on the drawing posttest. Again, a two-factorial analysis of covariance (ANCOVA) predicting learning outcome (drawing test score) with learner-generated drawing (yes/no) and author-generated picture (yes/no) as the factorial independent variables and prior knowledge as a covariate showed a significant positive main effect of learner-generated drawing, F(1, 163) = 62.60, p < .001, partial eta2 = .28, a significant positive main effect of author-generated pictures, F(1, 163) = 11.04, p = .001, partial eta2 = .06, and a significant interaction effect, F(1, 163) = 16.58, p < .001, partial eta2 = .09. In addition, multiple pairwise comparisons (with p < .05) showed that both the drawing group and the drawing + author-generated picture group performed significantly better than the author-generated picture group (d = .68; d = .59) and the control group (d = 1.87; d = 1.88). In turn, the author-generated picture group performed significantly better than the control group (d = .95). The drawing group and the drawing + author-generated picture group did not differ significantly from each other (d = .15).2 Overall, these results are consistent with Experiment 1 and provide additional support for the generative drawing effect. In accordance with Experiment 1, we were interested in whether differences in learning time among the experimental groups mediate the positive effect of drawing on text comprehension. First, an ANOVA predicting learning time with learner-generated drawing (yes/no) and author-generated picture (yes/no) as the factorial independent variables showed a significant main effect of learner-generated drawing, F(1, 164) = 392.26, p < .001, partial eta2 = .71, a significant main effect of author-generated picture, F(1, 164) = 16.85, p < .001, partial eta2 = .09, and a significant interaction effect F(1, 164) = 4.90, p = .028, partial eta2 = .03. Linear contrasts (with p < .05) revealed that the drawing group (M = 19.38 min, SD = 3.80) and the drawing + author-generated picture group (M = 23.40 min, SD = 5.51) needed significantly more learning time than the author-generated picture group (M = 9.60 min, SD = 3.97) and the control group (M = 8.34 min, SD = 2.51). Thus, to test whether learning time mediates the positive effect of drawing on text comprehension, additional mediation analyses (Baron & Kenny, 1986) were calculated by including learning time as an additional predictor in the aforementioned linear model. Results of the mediation analyses showed that the effects of drawing on the comprehension posttest and the drawing posttest scores (see multiple pairwise comparisons) are mediated by learning time to some extent. That is, including learning time in the linear model for predicting comprehension test scores still revealed a positive effect of the drawing group compared with the drawing + author-generated group on the comprehension test (p = .012). However, including learning time in the linear model for predicting comprehension posttest scores reduced the positive effect of the drawing group compared with the author-generated picture group (from p = .034 to p = .281) as well as compared with the control group (from p = .002 to p = .087), being 2 The exclusion of students’ prior knowledge in the reported analyses does not change the overall pattern of results. 284 A. Schmeck et al./Contemporary Educational Psychology 39 (2014) 275–286 no longer statistically significant. Regarding the drawing posttest score, including learning time does not change the reported pattern of results, except that the positive effect of the drawing + authorgenerated picture group compared with the author-generated picture group is no longer statistically significant (from p = .004 to p = .223). There were neither main effects of learner-generated drawing and author-generated pictures on the mental effort item (drawing group: M = 4.55, SD = 0.25; author-generated picture group: M = 4.59, SD = 0.24; drawing + author-generated picture group: M = 4.44, SD = 0.26; control group: M = 4.81, SD = 0.24; F < 1) nor on the perceived difficulty item (drawing group: M = 3.63, SD = 0.23; authorgenerated picture group: M = 3.71, SD = 0.22; drawing + authorgenerated picture group: M = 3.95, SD = 0.23; control group: M = 3.93, SD = 0.22; F < 1). Taken together, the drawing strategy apparently fosters students to engage in generative activities, indicated by their higher learning outcomes. Thus, the data provide further evidence for the generative drawing effect predicted by Schwamborn et al. (2010). In Experiment 2, benefits of the drawing activity, however, are mediated by learning time and do not involve higher mental effort. Additionally, there was no increased benefit when additional drawing support was available in the form of author-generated pictures. 3.4.4. Is there support for the prognostic drawing effect? A second major goal of this study was to determine whether the prognostic drawing effect could be extended to a new context. Mean proportion correct on drawing-accuracy during learning was .60 (SD = .04) for the drawing group and .68 (SD = .03) for the drawing + author generated picture group. This difference between the two drawing groups is not significant, F(1, 79) = 2.52, p = .116. This lack of group differences allowed us to pool the data of both drawing groups for subsequent correlation analyses. Correlation analyses based on the combined data from the two drawing groups revealed that the drawing-accuracy score of learner-generated drawings correlates significantly with the comprehension posttest score, r = .470, p < .001, as well as with the drawing posttest score, r = .615, p < .001. Additional correlation analyses revealed that the drawingaccuracy score of learner-generated drawings did not correlate significantly with the prior knowledge test score, r = .095, p = .400, the spatial ability test score, r = .127, p = .257, the motivation test score, r = .033, p = .769, or the mental effort test score, r = .042, p = .712. The correlation between the drawing-accuracy score and the perceived difficulty score was only slightly statistical significance, r = −.218, p = .053. Thus, the data provide further evidence for the prognostic drawing effect consistent with the results of Schwamborn et al. (2010). In sum, results of Experiment 2 are partly consistent with the results of Experiment 1 in that students learn better from a science text when they are asked to draw illustrations representing the main ideas of the text, and the quality of the generated drawings during learning correlates positively with students’ text comprehension (e.g., Schwamborn et al., 2010; van Meter, 2001; van Meter et al., 2006). 4. Discussion 4.1. Empirical contributions The present set of experiments makes three empirical contributions to the field. First, this study shows strong and consistent evidence that students who are asked to generate drawings (with sufficient support) during reading a scientific text that describes a causal sequence perform better than students who read without drawing, both on a comprehension test (d = 0.85 in Experiment 1 and d = 0.52 in Experiment 2) and on a drawing test (d = 1.15 in Experiment 1 and d = 1.87 in Experiment 2). Thus, the generative drawing effect can be extended to a new domain, and therefore corresponds to Shavelson and Towne’s (2002, p. 4) recommendation to “replicate and generalize across studies” as one of the six essential scientific principles of educational research. Second, this study shows strong and consistent evidence that the quality of drawings that students generate during learning with a scientific text that describes a causal sequence is positively related to subsequent performance on tests of learning outcome including a comprehension test (r = .623 in Experiment 1 and r = .470 in Experiment 2) and a drawing test (r = .620 in Experiment 1 and r = .615 in Experiment 2). Thus, the prognostic drawing effect can be replicated and extended to a new domain, consistent with standards for scientific research in education prescribed by Shavelson and Towne (2002). Third, this study shows that asking learners to draw pictures during reading a scientific text (i.e., learner-generated drawing group in Experiment 2) is more effective than simply providing drawings (i.e., author-generated picture group in Experiment 2) both on a comprehension test (d = 0.49) and a drawing test (d = 0.68). Similarly, adding author-generated drawings (i.e., learner-generated pictures + author-generated pictures group in Experiment 2) does not improve the learning outcomes of students who also draw pictures during learning (i.e., learner-generated pictures group in Experiment 2) either on a comprehension test (d = −0.57) or a drawing test (d = −0.15). In short, the act of drawing during learning (with sufficient support) improves learning beyond the simple provision of drawings. 4.2. Theoretical contributions The results are consistent with the idea that drawing during learning serves as a generative activity (Mayer & Wittrock, 2006; Schwamborn et al., 2010; van Meter & Garner, 2005; Wittrock, 1990). That is, the act of drawing encourages learners to engage in generative cognitive processing during learning such as organizing the relevant information into a coherent structure, and integrating it with relevant prior knowledge from long-term memory. In the present study, positive effects of drawing were indicated with a comprehension and a drawing learning outcome test, and therefore are in line with the theoretical assumption derived from the GTDC that benefits of drawing can be found if learning outcome tests are used that are sensitive to the underlying process of drawing (cf., van Meter & Garner, 2005). Additionally, in our study the drawing activity was supported in a way that was intended to help learners carry out the underlying cognitive processes of drawing (i.e., selecting, organizing and integrating) successfully. In this regard, results of the present study might supplement the theoretical framework of learnergenerated drawing by providing further evidence that benefits of drawing defined by van Meter and Garner’s GTDC can diminish, if no instructional support is given to constrain and structure the drawing activity. However, a fuller understanding of the underlying cognitive processes of drawing, and how these processes can be influenced via drawing support, requires more direct measures of cognitive processing during learning. Additionally, following the idea that metacognitive processes of monitoring and regulation are automatically activated by drawing (van Meter & Garner, 2005), a fuller understanding of the metacognitive effects of drawings is also required. 4.3. Practical contributions The present study encourages instructional designers and instructors to incorporate drawing activities into venues involving learning from text, which we call the generative drawing effect. One important feature of a successful drawing strategy that is present in this study and in a previous study by Schwamborn et al. (2010) is that the drawing activity was supported by providing a A. Schmeck et al./Contemporary Educational Psychology 39 (2014) 275–286 background scene and a legend showing how to represent each element to constrain and structure the drawing activity. Thus, an important practical implication is that students may need support in their drawing activity, so they do not need to draw from scratch. The present study also suggests a potentially useful diagnostic tool to gauge the depth of student learning, namely, the quality of the drawings created by students during learning, which we refer to as the prognostic drawing effect. Incorporating a measure of the quality of a learner’s drawing during learning can be a useful tool in developing remedial instruction to give learners individual support. It may be important to use materials that explain a cause-andeffect process and give learners drawings of the elements they need to represent the process pictorially. Asking learners to simply draw pictures of elements is unlikely to be helpful, whereas asking them to generate drawings that show the relations among the elements in a schematic form is more likely to be helpful. 4.4. Limitations and future directions Some limitations and future directions of our study should be addressed. As noted in the theoretical contributions subsection, we did not have direct measures of cognitive processing during learning; so it is not possible to pinpoint how the drawing activity affected specific cognitive processes such as attending to relevant information, organizing it, and integrating it with prior knowledge. We also did not assess metacognitive processing during learning, thus it is not possible to pinpoint how the drawing activity affected specific metacognitive processes such as monitoring and regulation. Furthermore, results of the cognitive load rating scales (invested mental effort and perceived task difficulty) are inconsistent. Whereas in Experiment 1 an effect on mental effort but not on perceived task difficulty showed up (i.e., students in the drawing group rated their invested mental effort during learning significantly higher), no effects on mental effort and task difficulty were found in Experiment 2. Additionally, in both experiments only a negative correlation of perceived task difficulty with the quality of learnergenerated pictures appears, but no correlation of mental effort with the quality. Following de Jong (2010) those cognitive load rating scales might have the disadvantages that they do not give a concurrent measure of cognitive load and do not measure an essential concept in cognitive load theory, namely cognitive overload (p. 125). Future studies on learner-generated drawing might also use other cognitive load measures, such as physiological measures as more direct indicators of cognitive load. As noted in the practical contributions subsection, we showed the drawing effects by using a scientific text describing how a causeand-effect system works, that is the causal steps regarding an infection with influenza and the immune response. It might be possible, however, that for other types of text, producing drawings might harm rather than promote text comprehension. Thus, to test whether the reported drawing effects can be extended, future research has to focus on other types of text such as descriptive texts as well as on other types of relations that can be conveyed with other types of representations, such as compare and contrast relations which can be shown in a matrix. Additionally, students’ learning outcomes were tested immediately after reading; thus, future work is needed to investigate the longer-term effects of generative drawing on learning outcomes. Furthermore, we only compared drawing with control groups that received no further learning strategy instructions. However, engaging in generative learning activities such as drawing requires a considerable amount of time. Accordingly, results showed that for Experiment 2, the positive effect of the drawing group on text comprehension compared with the author-generated picture group and to the control group was mediated by learning time. To rule out that the effects of drawing result only from additional time on task instead 285 of the generative activity future research should also compare the drawing strategy with other time demanding generative learning strategies such as summarization (cf., Leopold & Leutner, 2012). Another point that should be noted is that students in both experiments received some kind of multimedia materials in that, even when they had to draw and did not see presented pictures, they were at least provided with the basic (visual) elements for their drawings, which they had to do on the given background, which thus also contained information. In other words, when students are presented with important elements of the drawings, which they can use to draw themselves, they will not have to put as much effort into summarizing visually what they have just read compared with students who have to draw without any instructional help. Future studies might also compare the drawing group with a summarization group, in which students receive a set of verbal key terms that are similar to the drawing elements, and are asked to make a textual summary. Additionally, future research is needed to validate the prognostic drawing effect. So far, we know that the quality of learnergenerated pictures is related to students’ learning outcomes (i.e., the higher the learning outcome, the higher the drawing accuracy and vice versa) and their perceived difficulty (i.e., the lower the perceived difficulty the higher the drawing accuracy and vice versa), and that it is not related to students’ prior knowledge, motivation, spatial ability or mental effort. However, less is known about what this might mean. That is, less is known regarding the causal direction of this relation or the presence of a possible further moderator variable. Do students’ efforts to produce accurate drawings lead to better comprehension and lower perceived difficulty? Or do students who are more adept in drawing benefit more from the strategy and thus perceive the difficulty of the learning materials as being lower? Both arguments seem convincing. Finally, more work is needed to determine the level of support that makes the drawing strategy most effective for various kinds of learners. As noted in the empirical contribution adding authorgenerated drawings (i.e., learner-generated pictures + authorgenerated pictures group in Experiment 2) does not improve the learning outcomes of students who also draw pictures during learning and were supported by a drawing prompt. In other words, the combination of two ways of supporting the drawing strategy (i.e., giving a drawing prompt during reading plus an author-generated picture after reading) did not improve students’ learning outcomes compared with students in the drawing group as well as compared with students in the control and author-generated pictures only groups. This result is inconsistent with previous research (e.g., van Meter, 2001; van Meter et al., 2006) which found that comparing own drawings to author-generated pictures normally helps learning. van Meter and colleagues (2001, 2006), however, provided author-generated pictures plus prompting questions after the drawing process. That is, students answered prompting questions to guide the comparison process between their self-generated drawing and the author-generated drawing. In our study, students were only instructed to generate a drawing, to inspect an authorgenerated one, and to check whether their own drawing in comparison with the author-generated one really represented the main ideas of the text paragraph correctly. In other words, we did not guide the process of comparing self-generated drawings with author-generated ones. As a potential consequence students performed the intended comparison process inadequately or even not all, and thus did not benefit from it. One reason for this inadequate comparison process might be that students need guidance in doing the comparison process. Another reason might be the fact that students do not seriously engage in generating drawings once they notice that there are author-generated drawings. Thus, future research should also use additional guidance to test whether the combination of different ways of supporting the drawing strategy 286 A. Schmeck et al./Contemporary Educational Psychology 39 (2014) 275–286 (i.e., giving a drawing prompt during reading plus an authorgenerated picture after reading) helps learning as well as observational measures of the drawing process itself (i.e. think aloud protocols) to shed more light on the cognitive processes underlying the drawing activities. Overall, drawing during learning from text appears to be a potentially powerful strategy for improving students’ learning from scientific text when certain boundaries and prerequisites are taken into account. Acknowledgments This article is based on a research project funded by the German Research Foundation (DFG; LE 645/9-3 as part of FOR 511). We would like to thank Angela Sandmann for her assistance in developing the learning materials. References Ainsworth, S., Prain, V., & Tytler, R. (2011). Drawing to learn in science. Science, 333, 1096–1097. Alesandrini, K. L. (1981). Pictorial-verbal and analytic-holistic learning strategies in science learning. Journal of Educational Psychology, 73, 358–368. Alesandrini, K. L. (1984). Pictures and adult learning. Instructional Science, 13, 63–77. Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173–1182. Brünken, R., Plass, J. L., & Leutner, D., (2003). Direct measurement of cognitive load in multimedia learning. Educational Psychologist, 38, 53–61. Carney, R.N., & Levin, J.R. (2002). Pictorial illustrations still improve students’ learning from text. Educational Psychology Review, 14, 5–26. de Jong, T. (2005). The guided discovery principle in multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 215–228). New York: Cambridge University Press. de Jong, T. (2010). Cognitive load theory, educational research, and instructional design: Some food for thought. Instructional Science, 38, 105–134. Ekstrom, R. B., French, J. W., & Harman, H. H. (1976). Manual for kit of factor-referenced cognitive tests. Princeton, NJ: Educational Testing Service. Greene, T. R. (1989). Children’s understanding of class inclusion hierarchies: The relationship between external representation and task performance. Journal of Experimental Child Psychology, 48, 62–89. Hall, V. C., Bailey, J., & Tillman, C. (1997). Can student-generated illustrations be worth ten thousand words? Journal of Educational Psychology, 89, 677–681. Höffler, T. N. (2010). Spatial ability: Its influence on Learning with visualizations – A meta-analytic review. Educational Psychology Review, 22, 245–269. Höffler, T. N., Schmeck, A., & Opfermann, M. (2013). Static and dynamic visual representations: Individual differences in processing. In G. Schraw, M. T. McCrudden, & D. Robinson (Eds.), Learning through visual displays (pp. 133–163). Charlotte, NC: Information Age Publishing. Kalyuga, S., Chandler, P., & Sweller, J. (1999). Managing split-attention and redundancy in multimedia instruction. Applied Cognitive Psychology, 13, 351–371. Leopold, C. (2009). Lernstrategien und Textverstehen [Learning strategies and text comprehension]. Münster: Waxmann. Leopold, C., & Leutner, D. (2012). Science text comprehension: Drawing, main idea selection, and summarizing as learning strategies. Learning and Instruction, 22, 16–26. Lesgold, A. M., DeGood, H., & Levin, J. R. (1977). Pictures and young children’s prose learning: A supplementary report. Journal of Reading Behavior, 9, 353–360. Lesgold, A. M., Levin, J. R., Shimron, J., & Guttman, J. (1975). Pictures and young children’s learning from oral prose. Journal of Educational Psychology, 67, 636–642. Leutner, D., Leopold, C., & Sumfleth, E. (2009). Cognitive load and science text comprehension: Effects of drawing and mentally imagining text content. Computers in Human Behavior, 25, 284–289. Mayer, R. E. (2004). Should there be a three-strikes rule against pure discovery learning? The case for guided methods of instruction. The American Psychologist, 59, 14–19. Mayer, R. E. (2005). Cognitive theory of multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 31–48). New York: Cambridge University Press. Mayer, R. E. (2009). Multimedia learning (2nd ed.). New York, NY: Cambridge University Press. Mayer, R. E., & Wittrock, M. C. (2006). Problem solving. In P. Alexander, P. Winne, & G. Phye (Eds.), Handbook of educational psychology (pp. 287–303). Mahwah, NJ: Erlbaum. Paas, F. (1992). Training strategies for attaining transfer of problem-solving skill in statistics—A cognitive-load approach. Journal of Educational Psychology, 84, 429–434. Paas, F., Tuovinen, J., Tabbers, H. K., & Van Gerven, P. W. M. (2003). Cognitive load measurement as a means to advance cognitive load theory. Educational Psychologist, 38, 63–71. Paivio, A. (1986). Mental representation: A dual coding approach. New York: Oxford University Press. Pashler, H., Bain, P., Bottage, B., Graesser, A., Koedinger, K., McDaniel, M., et al. (2007). Organizing instruction and study to improve student learning. Washington, DC: National Center for Educational Research. Rasco, R. W., Tennyson, R. D., & Boutwell, R. C. (1975). Imagery instructions and drawings in learning prose. Journal of Educational Psychology, 67, 188–192. Rheinberg, F., Vollmeyer, R., & Burns, B. D. (2001). FAM: Ein fragebogen zur erfassung aktueller motivation in lern- und leistungssituationen [QCM: A questionnaire to assess current motivation in learning situations]. Diagnostica, 47, 57–66. Schnotz, W. (2005). An integrated model of text and picture comprehension. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 49–70). New York: Cambridge University Press. Schwamborn, A., Mayer, R. E., Thillmann, H., Leopold, C., & Leutner, D. (2010). Drawing as a generative activity and drawing as a prognostic activity. Journal of Educational Psychology, 102, 872–879. Schwamborn, A., Thillmann, H., Opfermann, M., & Leutner, D. (2011). Cognitive load and instructionally supported learning with provided and learner-generated visualizations. Computers in Human Behavior, 27, 89–93. Shavelson, R. J., & Towne, L. (Eds.), (2002). Scientific research in education. Washington, DC: National Academy Press. Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive Load Theory. New York: Springer. Tirre, W. C., Manelis, L., & Leicht, K. (1979). The effects of imaginal and verbal strategies on prose comprehension by adults. Journal of Reading Behavior, 11, 99–106. van Meter, P. (2001). Drawing construction as a strategy for learning from text. Journal of Educational Psychology, 69, 129–140. van Meter, P., Aleksic, M., Schwartz, A., & Garner, J. (2006). Learner-generated drawing as a strategy for learning from content area text. Contemporary Educational Psychology, 31, 142–166. van Meter, P., & Garner, J. (2005). The promise and practice of learner-generated drawings: Literature review and synthesis. Educational Psychology Review, 12, 261–312. Van Gog, T., & Paas, F. (2008). Instructional efficiency: Revisiting the original construct in educational research. Educational Psychologist, 43, 16–26. Vollmeyer, R., & Rheinberg, F. (2000). Does motivation affect learning via persistence? Learning and Instruction, 4, 293–309. Weinstein, C. E., & Mayer, R. E. (1986). The teaching of learning strategies. In M. C. Wittrock (Ed.), Handbook of research on teaching (pp. 315–327). New York: Macmillan. Wittrock, M. C. (1990). Generative processes of comprehension. Educational Psychologist, 24, 345–376.