Observational Learning in Instructional Second Language Writing Rashmie Bhagwan-Snetselaar Studentnumber: 0114731 Educational Master Thesis Supervisor: Huub van den Bergh 22 February 2008 Table of Contents Abstract 3 1. Introduction 4 2. Method 7 3. Results 10 4. Conclusion and Discussion 15 5. References 16 Appendix 17 2 Abstract Studies have shown positive effects for observational learning concerning writing in a first language. This study investigates whether observational learning as teaching method can also be successfully applied to writing in a second language. An experiment was conducted in which forty students wrote experiment manuals. These manuals were executed by peers. This was videotaped. After seeing these video recordings the students revised their versions. There were two observational learning conditions. One group of students observed readers of their own manual. The other group of students observed readers of their peer’s manuals. The results show significant effects of observational learning. The students performed better on the revised version. No significant effects were found concerning the observational learning conditions. It does not matter whether students observe readers of their own manual or someone else’s manual. It does not have any effect on the improvement of the manual. 3 1. Introduction Writing is an essential part in language education. When students are faced with a writing task they are actually faced with two tasks. The first task is the writing task and the second task is the task of learning how to write (Rijlaarsdam & Braaksma, 2004). Learning how to write requires metacognitive activities such as planning, monitoring, reflection, etc. According to Rijlaarsdam and Braaksma (2004), weak learners are not able to learn how to write because they can only focus on one task which is the writing task. This task is a complex task and a lot of effort is needed in order to produce a written text. Two tasks will overload the working memory capacity. Furthermore, assignments in school books often only focus on the writing task because these assignments are product-oriented. The learning how to write task is barely given attention to. One way to stimulate the use of metacognitive activities is revision. By revising texts, writers can look at their writing as readers and reflect on their own texts. However, possible errors are hard to detect because the writer still has in mind what he thought he wrote. By rereading the text the writer “can perceive dissonance between intention and production” (Bereiter & Scardamelia, 1987). It is therefore better to revise texts written by someone else. Writers still act as readers by using peer revision. They give comments on the writing of peers and transfer this to their own writing (Hillocks, 1986). Despite of the metacognitive activities revision or peer revision stimulate, the use of revision remains product-oriented because it focuses on the outcome of the task. No attention is paid to the writing process. Moreover, the revisions that are made are often surface revisions such as spelling and punctuation changes (Fitzgerald, 1987). This means that the focus of revision is on the language of a writing product rather than the meaning. This is true for novice writers or less competent writers. Studies have shown that novice writers see revision merely as correcting errors whereas expert writers see revision as rethinking. (Flower & Hayes 1986; Sommers 1980). In other words, expert writers make more meaning changes and novice writers make more surface changes. Furthermore, in some studies the quality of a text deteriorated after revision (Bereiter and Scardamelia, 1987; Breetveld, van den Bergh & Rijlaarsdam 1994). A method that is process-oriented is called observational learning. Observational learning has been successfully applied to several subjects in education including writing. Other subjects are, for example, mathematics and reading. The most striking feature of observational learning concerning writing is that learners are not only occupied with the writing task. By observing others perform a writing task they do not write themselves which gives them the opportunity to focus on the learning task. The working memory is not overloaded since observing is a less complex task than producing a written text. Learners can devote their working memory capacity to learn how to write and retrieve this knowledge to produce a written text. Couzijn (1995) conducted two studies in which the positive effects of observational learning were shown. In the first study participants were taught how to perform a simple physics experiment. The participants were asked to write a manual about the experiment. Other students had to perform the experiment by means of the participants’ manuals. Some of these students also gave written comments about how to improve the manual. These performances were videotaped and shown to the writers subjected to the observational learning conditions. Three conditions of observational learning were measured. In the first condition writers observed readers of their own manuals and also received written feedback. In the second condition writers only observed readers of their own manuals. In the third condition writers observed readers of their peers’ manuals. After being confronted with the writing by seeing 4 the videotapes, the writers received their original manuals and were asked to revise it. Writers in the control condition were only asked to revise their manuals. The results showed that the manuals in the control condition did not significantly improve. Moreover, all three observational learning conditions scored significantly better than the control condition. The condition in which the writers received additional feedback scored the best. Three weeks later the participants were asked to write a letter of advice to peers concerning manuals. Again the observational learning conditions scored better than the control condition. These results show that the observing writers did not only improve their writing but also learned about how to write a manual. This proves that observational learning contributes to the learning how to write task. In his second study Couzijn (1995) conducted a study concerning argumentative writing. The participants were fifteen year old students. All students were provided with information and explanation about argumentative writing. There were two conditions. In the first condition students had to do exercises about argumentative writing in their books. The students in the second condition did not do the exercises. They watched videotapes instead in which two pupils were doing exercises. The results showed that the observing students could write texts better than the students who had done the exercises. The observing students could also analyse texts better than the students who did exercises about analysing texts. This shows that learning by observing yields better results than learning by doing. Couzijn’s studies focus on the writing outcomes of observational learning and not the underlying processes of observational learning. To get more insight into the underlying processes of observational learning Braaksma (2002) continuated Couzijn’s study regarding argumentative writing. In her thesis Braaksma focuses on three issues concerning the underlying processes of observational learning. The first issue addresses the observational process. Braaksma examined which aspects of observational learning contributes to learning gains. To examine this Braaksma used a post-hoc analysis from Couzijn’s data. Three learning activities were found: evaluation, product elaboration and process elaboration. By evaluating and elaborating on tasks that others have performed observing learners can write better texts. The second issue concerns the type of instruction for observational learning and the type of learner. An experiment was conducted in which participants were given instructions about argumentative writing. There were two observational learning conditions and one control condition. In both observational learning conditions participants observed two models performing writing tasks. In one observational learning condition participants were asked to focus on the weak model and in the other observational learning condition participants received instructions to focus on the good model. The participants in the control condition only performed writing tasks. Results indicated that weak learners gained more from focussing on the weak model whereas good learners learned more from focussing on good models. This was referred to as the model observer similarity. The third issue Braaksma addressed concerns the changes which occur in the writing process due to observational learning. The conditions in this study were the same as the previous study. Participants were either asked to focus on the weak model or the good model. After this the participants performed writing tasks in which they wrote short argumentative texts. In order to get insight into the writing process the participants were asked to think aloud. The results showed that participants in the observational learning condition used more metacognitive activities than the participants in the control condition. Moreover, participants in the observational learning condition showed more analysing activities at the beginning of the writing process than at the end. Participants in the control condition showed a monotonous process. 5 In sum, the studies of Braaksma have shown that learners evaluate and elaborate when they observe writing activities. Moreover, observational learning stimulates the use of metacognitive activities. This shows that learners are able to concentrate on both the writing task as the learning how to write task. Moreover, observational learning gives weak learners the opportunity to learn from the writing task when they observe weak models. As aforementioned, observational learning has been successfully applied to various subjects in education such as mathematics and reading. Couzijn and Braaksma’s studies have shown that observational learning has positive effects on the subject writing. These studies concern writing in a first language. An interesting question is whether observational learning also yields better results for writing in a second language. If the answer to this question is positive then observational learning could be an effective teaching method for all subjects taught in school. Therefore, the present study focuses on observational learning with respect to writing in a second language. The type of writing chosen for the present study is instructional writing. This type of writing does not occur much in education which is disappointing because instructional writing has a strong communicative aspect. In addition, more than any other type of text, instructional writing requires a strong awareness of the reader (Maes, Ummelen & Hoeken 1996). Two questions are addressed in the following study. The first question concerns the effectiveness of observational learning in second language instructional writing. In other words, is observational learning in second language instructional writing effective? The second question concerns the observing method used for observational learning. Is observing a reader of your own text more effective than observing a reader of a text written by someone else? 6 2. Method Subjects Two nine grade classes (highest level) of a secondary school (havo/vwo) participated in the study. One class consisted of twenty students and the other class consisted of twenty-one students. At first, forty students participated in the study. One student was absent during the first session. This student was excluded from the initial sample. Of the remaining forty students, twenty-one were females and nineteen were males. The students were randomly assigned to two conditions: observing-video-own-text and observing-video-other text. Another twenty pupils took part in the study by being videotaped while conducting an experiment. Materials The subjects were asked to write a manual for a simple science experiment called cork popping. The experiment cork popping shows that a cork pops off a bottle filled with baking soda and lemon juice. When these two ingredients are mixed a chemical reaction occurs. Carbon dioxide is produced which creates pressure. As the pressure builds the cork pops off the bottle. The teacher conducted the experiment in front of the class and explained why and what had happened. Afterwards, the students wrote a first version of their manual. The manuals were handed in and the teacher typed out the manuals. After the students wrote the manuals other students were asked to conduct the experiment by means of the manuals written by the subjects. This was videotaped. The subjects were randomly assigned to two conditions. The subjects assigned to the observing video own text condition observed readers of their own manuals. The subjects assigned to the observing video other text condition observed readers of their peers’ manuals. After the subjects observed real readers of their own or their peer’s manuals they were asked to write a revised version of their manuals. In order to compare the first and revised version of the manuals nine aspects of writing a good manual of the cork popping experiment were scored. Table 1 shows these aspects and the scoring procedure. The first three aspects are related to general aspects of a good manual. Aspects four and five are needed in order to perform the experiment correctly. When certain ingredients or steps are not mentioned or not mentioned correctly the experiment will fail. Aspects six and seven guide the reader in terms of feedback or result. Examples of comments to the reader are ‘be careful’ and ‘make sure you have read all the instructions before conducting the experiment’. The last two aspects concerns the use of the English language. Language errors can lead to failure of the experiment because the manual is incomprehensible. It should be noted that the language aspect style was not measured since vocabulary knowledge of nine grade second language learners is small. 7 Table 1. Aspects of a manual and scoring procedure _____________________________________________________________ aspect of manual scoring procedure _____________________________________________________________ 1. 2. 3. 4. list of the things needed mentioned yes (0) or no (1) instructions step by step mentioned yes (0) or no (1) explanation outcome mentioned yes (0) or no (1) number of ingredients - a bottle mentioned yes (0) or no (1) - a cork mentioned yes (0) or no (1) - water mentioned yes (0) or no (1) - 2 lemons mentioned yes (0) or no (1) - baking soda mentioned yes (0) or no (1) - piece of paper mentioned yes (0) or no (1) - teaspoon mentioned yes (0) or no (1) 5. number of steps - squeeze lemons mentioned yes (0) or no (1) - pour juice in bottle mentioned yes (0) or no (1) - add water mentioned yes (0) or no (1) - shake bottle mentioned yes (0) or no (1) - baking soda on paper mentioned yes (0) or no (1) - soda in bottle, cork on mentioned yes (0) or no (1) - wait until cork pops mentioned yes (0) or no (1) 6. comments reader mentioned yes (0) or no (1) 7. number of comments (number per 100 words) 8. number of spelling errors (number per 100 words) 9. number of grammatical (number per 100 words) errors _____________________________________________________________ Procedure Participation in the study took place in two sessions during two English lessons in each class. In the first session the students were asked to write a manual in English for the cork popping experiment after they had seen the teacher conducting the experiment in front of the class. The demonstration of the experiment by the teacher took approximately ten minutes. During the demonstration the students were allowed to make notes. After the demonstration the students were given time to write the manuals individually in the remaining time of the lesson. Some students handed in their manuals after fifteen minutes others after thirty minutes. The students were allowed to use a dictionary while writing their manuals. Before the second session took place the teacher had typed out the manuals and other students had conducted the experiment by using the subjects’ manuals while being videotaped. These students were given instruction to think aloud. During the second session two students were given back their manuals and a videotape was shown. One of the two students observed a reader of his/her own manual. The other student observed a reader of his/her peer’s manual. This was done ten times because there were twenty participants in each class. After seeing the videotape the students were asked to revise their first version. A total of eighty manuals were scored. There were two observers scoring the data. The reliability is high as seen in table 2. For further analyses the average scores of the two observers were used. 8 Table 2. Reliability per variable and version _________________________________________________________ Cronbach’s alpha _________________________________________________________ Variable First version Revised version _________________________________________________________ list of the things needed 1.00 0.84 instructions step by step 1.00 0.84 explanation outcome 1.00 0.78 amount of steps 0.97 0.96 amount of ingredients 1.00 1.00 comments reader 0.98 0.97 number of comments 0.99 0.98 number of spelling errors 0.99 0.94 number of grammatical errors 1.00 0.91 _________________________________________________________ 9 3. Results Descriptive statistics for the first and revised version of the experiment manual were calculated per dependent variable. Descriptive statistics were also calculated per condition. These data were tested for significant differences using regression analyses. Descriptive statistics for the variable list of the things needed are presented in table 3. On average, the students scored better on the revised version than on the first version. Furthermore, students from the condition observing video other text scored better on the revised version. The scores of condition observing video own text remained the same. Table 3. Descriptive statistics for the variable list of the things needed (mean, standard deviation) ___________________________________________________________ First version Revised version ___________________________________________________________ Condition M SD M SD ___________________________________________________________ observing video other text 0.80 0.41 0.85 0.37 observing video own text 0.85 0.37 0.85 0.33 ___________________________________________________________ No significant effect was found between both versions (F = 0.66; df = 1, 38; p = .423). It cannot be concluded that the mean differs between both versions. Also, no significant interaction effect of list of the things needed and condition observing video own text was found (F = 0.66; df = 1, 38; p = .423). In table 4 the descriptive statistics for the variable instructions step by step are presented. The students scored better on the revised version than on the first version. Furthermore, students from both conditions scored better on the revised version. Table 4. Descriptive statistics for the variable instructions step by step (mean, standard deviation) ____________________________________________________________ First version Revised version ____________________________________________________________ Condition M SD M SD ____________________________________________________________ observing video other text 0.60 0.50 0.80 0.41 observing video own text 0.70 0.47 0.73 0.44 ____________________________________________________________ A significant effect was found between both versions. The students wrote their instructions more in steps on the revised version (F = 5. 60; df = 1, 38; p < .05). No significant interaction effect of instructions step by step and condition observing video own text was found ( F = 3.39; df = 1, 38; p = .074). 10 Descriptive statistics for the variable explanation outcome are presented in table 5. The students scored slightly better on the revised version than on the first version. Furthermore, students from the condition observing video own text scored better on the revised version. The scores of condition observing video other text remained the same. Table 5. Descriptive statistics for the variable explanation outcome (mean, standard deviation) ___________________________________________________________ First version Revised version ___________________________________________________________ Condition M SD M SD ___________________________________________________________ observing video other text 0.90 0.31 0.90 0.31 observing video own text 0.95 0.22 0.98 0.11 ___________________________________________________________ No effect was found between both versions (F = 0.19; df = 1, 38; p = .664). The mean scores do not differ significantly. Furthermore, no significant interaction effect of explanation outcome and condition observing video own text was found (F = 0.19; df = 1, 38; p = .664). In table 6 the descriptive statistics for the variable number of ingredients are presented. The students scored better on the revised version than on the first version. Furthermore, students from both conditions scored better on the revised version. Table 6. Descriptive statistics for the variable number of ingredients (mean, standard deviation) ___________________________________________________________ First version Revised version ___________________________________________________________ Condition M SD M SD ___________________________________________________________ observing video other text 6.85 0.37 6.95 0.22 observing video own text 6.50 0.51 6.80 0.41 ___________________________________________________________ A significant effect was found between both versions. The pupils scored better on the revised version. (F = 7. 60; df = 1, 38; p < .01). No interaction effect of number of ingredients and condition observing video own text was found( F = 1.90; df = 1, 38; p = .176). The mean scores do not differ significantly. The descriptive statistics for the variable number of steps are presented in table 7. The students scored better on the revised version than on the first version. Furthermore, students from both conditions scored better on the revised version. 11 Table 7. Descriptive statistics for the variable number of steps (mean, standard deviation) ___________________________________________________________ First version Revised version ___________________________________________________________ Condition M SD M SD ___________________________________________________________ observing video other text 6.10 0.74 6.15 0.65 observing video own text 6.35 0.67 6.50 0.51 ___________________________________________________________ No significant effect was found between both versions (F = 2.03; df = 1, 38; p = .163). Furthermore, no significant interaction effect of number of steps and condition observing video own text was found ( F = 0.51; df = 1, 38; p = .481). In table 8 the descriptive statistics for the variable comments reader are presented. The students scored better on the revised version than on the first version. Furthermore, students from both conditions scored better on the revised version. Table 8. Descriptive statistics for the variable comments reader (mean, standard deviation) ___________________________________________________________ First version Revised version ___________________________________________________________ Condition M SD M SD ___________________________________________________________ observing video other text 0.43 0.49 0.53 0.41 observing video own text 0.50 0.51 0.55 0.48 ___________________________________________________________ No effect was found between both versions (F = 3.80; df = 1, 38; p = .059). The mean scores do not differ significantly. Also, no significant interaction effect of comments reader and condition observing video own text was found ( F = 0.42; df = 1, 38; p = .520). In table 9 the descriptive statistics for the variable number of comments are presented. The students scored better on the revised version than on the first version. Furthermore, students from both conditions scored better on the revised version. 12 Table 9. Descriptive statistics for the variable number of comments (mean, standard deviation) ____________________________________________________________ First version Revised version ____________________________________________________________ Condition M SD M SD ____________________________________________________________ observing video other text 0.58 0.75 0.90 0.74 observing video own text 0.65 0.75 0.80 0.77 ____________________________________________________________ A significant effect was found between both versions. In other words, the students used more reader comments in the revised version. (F = 5. 47; df = 1, 38; p < .05). No significant interaction effect of number of comments and condition observing video own text was found ( F = 0.74; df = 1, 38; p = .394). In table 10 the descriptive statistics for the variable number of spelling errors are presented. The students scored better on the revised version than on the first version. Furthermore, students from both conditions scored better on the revised version. Table 10. Descriptive statistics variable number of spelling errors (mean, standard deviation) ___________________________________________________________ First version Revised version ___________________________________________________________ Condition M SD M SD ___________________________________________________________ observing video other text 1.28 1.25 1.05 1.20 observing video own text 1.46 0.91 1.10 0.80 ___________________________________________________________ A significant effect was found between both versions. The students reduced the amount of spelling errors on the revised version (F = 13.46; df = 1, 38; p < .001). No significant interaction effect of spelling errors and condition observing video own text was found ( F = 0.62; df = 1, 38; p = .436). The descriptive statistics for the variable number of comments are presented in table 11. The students scored better on the revised version than on the first version. Furthermore, students from both conditions scored better on the revised version. 13 Table 11. Descriptive statistics for the variable number of grammatical errors (mean, standard deviation) _______________________________________________________________ First version Revised version _______________________________________________________________ Condition M SD M SD _______________________________________________________________ observing video other text 2.22 1.67 1.74 1.42 observing video own text 3.21 2.74 2.20 2.39 _______________________________________________________________ A significant effect was found between both versions. The students reduced the amount of grammatical errors on the revised version (F = 14.38; df = 1, 38; p < .001). No significant interaction effect of grammatical errors and condition observing video own text was found ( F = 1.82; df = 1, 38; p = .185). The descriptive statistics per gender for each dependent variable and for both versions of the manual are presented in appendix A. On average, females scored better on both versions than males. On the other hand, females made more language errors than males. However, no effect was found concerning gender. The mean scores do not differ significantly as shown in appendix B. 14 4. Conclusion and Discussion The first question raised in this study is whether observational learning in second language writing is effective. The scores on the first and revised version of the experiment manual differed significantly for five out of nine variables. The students improved their own manuals by using steps to give instructions to the reader. They also mentioned more ingredients. Furthermore, more comments to the reader were added. Lastly, spelling and grammatical errors were reduced. Overall, observational learning yields better results. Thus, observational learning in second language writing is effective. The second question concerns the effectiveness of observational learning due to observing own texts. No significant interaction effects were found between the observing video own text condition and the dependent variables. It does not matter whether students observe readers of their own manual or someone else’s manual. It does not have any effect on the improvement of the manual. It should be noted that the manuals in this study were hard to improve because of ceiling effects. The students already scored high in their first versions. For example, out of seven ingredients the students already mentioned six in their first version. This could imply that the experiment was too easy. On the other hand, the experiments students conducted by means of the participants’ manuals did not all succeed. In only one out of forty experiments did the cork pop off the bottle. In her study Braaksma concluded that weak learners gained more from focussing on weak models whereas good learners learned more from focussing on good models. This model observer similarity does not apply to the present study as all the learners improved their writing on the revised version. An interesting question is whether this is due to the second language. Further research should investigate this. Aside from observing the participants in Braaksma’s study were also asked to answer the questions what went well and what did not went well in the videotapes. This study did not take this into account. In further research this should be considered. Further research on observational learning in second language writing should also take into account the following things. Firstly, a control group should be added to the conditions to investigate whether observational learning conditions yields better results in comparison to revising only. Secondly, teacher comments as model could be added to see whether observational learning is effective compared to using teacher comments as model. Thirdly, this study focussed on writing outcomes of observational learning. The underlying processes of observational learning regarding second language writing should also be investigated in further research. In conclusion, this study has shown that observational learning concerning writing can also be applied to a second language. Using observational learning as a teaching method in education may seem time-consuming at first, but this study has also shown that one single model will suffice. A teacher can implement observational learning by using one tape recording of a student performing an experiment, for example, before students revise their writing. 15 5. References Bereiter, C., & Scardamalia, M. (1987). The psychology of written composition. Hillsdale, NJ: Lawrence Erlbaum. Braaksma, M. A. H. (2002). Observational learning in argumentative writing. Proefschrift Universiteit van Amsterdam. Breetvelt, I., Van den Bergh, H., & Rijlaarsdam, G. (1994). Relations between writing processes and text quality: When and how? Cognition and Instruction, 12, (2), 103-123. Couzijn, M. J. (1995). Observation of writing and reading activities. Effects on learning and transfer. Proefschrift. Universiteit van Amsterdam. Fitzgerald, J. (1987). Research on Revision in Writing. Review of Educational Research, 57, ( 4), 481-506. Flower, L. S., & Hayes, J. R. (1980). The dynamics of composing: Making plans and juggling constraints. In L. W. Gregg & E. R. Steinberg (Eds.), Cognitive processes in writing: An interdisciplinary approach. 31-50. Hillsdale, NJ: Lawrence Erlbaum. Hillocks, G. (1986). Research on written composition: New directions for teaching. Urbana, Ill.: ERIC Clearinghouse on Reading and Communication Skills, National Institute of Education. Maes, A., Ummelen, N. & H. Hoeken. (1996). Instructieve teksten: analyse, ontwerp en evaluatie. Bussum: Coutinho. Rijlaarsdam, G.C.W., & Braaksma, M.A.H. (2004). Schrijven en leren schrijven. Niet zelf doen, maar zien hoe anderen het doen. Levende Talen Magazine 3, 17-22. Sommers, N. (1980). Revision strategies of student writers and experienced adult writers. College Composition and Communication, 31, 378-388. 16 Appendix A Descriptive statistics (mean, standard deviation) concerning gender per variable for the first and revised version. ________________________________________________________________ first version revised version ________________________________________________________________ variable gender mean SD mean SD ________________________________________________________________ list of ingredients male 0.74 0.45 0.79 0.38 female 0.90 0.30 0.90 0.30 manual in steps male 0.47 0.51 0.55 0.50 female 0.81 0.40 0.95 0.22 explanation male 0.90 0.32 0.92 0.25 female 0.95 0.22 0.95 0.22 amount of steps male 6.16 0.85 6.21 0.68 female 6.29 0.56 6.43 0.51 amount of ingredients male 6.68 0.48 6.95 0.23 female 6.67 0.48 6.81 0.40 comments reader male 0.45 0.50 0.47 0.46 female 0.48 0.51 0.54 0.44 number of comments male 0.55 0.69 0.66 0.67 female 0.67 0.80 0.66 0.67 number of spelling errors male 1.19 0.95 1.02 1.03 female 1.53 1.19 1.13 1.01 number of grammatical errors male 1.56 1.33 1.23 1.25 female 3.75 2.50 2.65 2.24 ________________________________________________________________ 17 Appendix B Effect concerning gender per variable _________________________________________________ variable effect _________________________________________________ list of ingredients F = 0.65; df = 1, 37; p = .424 manual in steps F = 0.58; df = 1, 37; p = .452 explanation F = 0.23; df = 1, 37; p = .635 amount of steps F = 0.36; df = 1, 37; p = .551 amount of ingredients F = 0.80; df = 1, 37; p = .376 comments reader F = 1.55; df = 1, 37; p = .220 number of comments F = 1.68; df = 1, 37; p = .204 number of spelling errors F = 1.85; df = 1, 37; p = .183 number of grammatical errors F = 3.90; df = 1, 37; p = .056 _________________________________________________ 18