1 Assessment of Critical Thinking through Writing an English Argumentative Essay A Research-Based Paper Submitted to the School of Postgraduate Studies English Department Program in Partial Fulfillment of the Requirements for the Final Test of Language Testing and Assessment Disusun Oleh : DEDEH ROHAYATI STUDENT NUMBER: 1201025 CLASS 3A ENGLISH EDUCATION DEPARTMENT SCHOOL OF POSTGRADUATE STUDIES INDONESIA UNIVERSITY OF EDUCATION 2 Assessment of Critical Thinking through Writing an English Argumentative Essay Dedeh Rohayati Indonesia University of Education rohayatidedeh@rocketmail.com Abstract In connection with critical thinking assessment which is still problematic, many experts propose various techniques to accommodate it. They believe that among other things, writing argumentative essay can be applied to assess critical thinking ability. Thus, the use of Brookhart’s rubric seems to be applicable to assess three aspects of critical thinking such as thesis, evidence, and reasoning and clarity. Supported by these notions, this study is therefore aimed to report how to assess critical thinking through writing argumentative essay and to investigate the students’ responses. By utilizing qualitative design and a simple descriptive statistic, six students’ English argumentative essays are collected. The result shows that the students are able to state these aspects; yet, they are poor in expressing accurate evidence of the phenomenon. Overall, their critical thinking ability needs improving. On the other hand, based on questionnaires answers, this study, to a great extent, reveals that critical thinking assessment is beneficial to be conducted because it is considered as a feedback for student. In addition, it can motivate them to have a better critical thinking. To be concluded, this study appears to be good evidence that critical thinking assessment can be administered through writing. Evidently, it is recommended that both sustainable teaching critical thinking and its assessment should be implemented in the classroom effectively. Key words: assessment, critical thinking, writing argumentative essay Abstrak Berkaitan dengan penilaian berpikir kritis yang masih problematik, para ahli di bidangnya mengusulkan berbagai tekhnik diantaranya melalui tulisan argumentative esei. Sehingga, penggunnaan rubric Brookhart nampaknya dapat diaplikasikan untuk menilai aspek-aspek berpikir kritis seperti tesis (thesis), pembuktian (evidence), alasan (reason) dan kejelasan (clarity). Pemikiran tersebut telah mendorong penelitian ini dilakukan dengan tujuan untuk memaparkan penilaian berpikir kritis melalui tulisan esei argumentative sekaligus untuk mengetahui responnya dari para mahasiswa. Penelitian kualitative yang menggunakan statistik sederhana ini, menggunakan 6 argumentative esei bahasa Inggris. Hasil penelitian menunjukan bahwa mereka mampu menyatakan semua aspek-aspek berpikir kritis tersebut, namun belum maksimal dalam mengungkapkan pembuktian-pembuktian yang akurat. Sedangkan dari hasil jawaban kuesioner, secara garis besar mereka merasakan pentingnya penilaian ini sebagai umpan balik yang dapat memotivasi mereka untuk memiliki kemampuan berpikir kritis yang lebih baik. Kesimpulannya, hasil penelitian ini menjadi suatu bukti bahwa penilaian berpikir kritis dapat dilakukan berdasarkan hasil tulisan..Terakhir, hasil kajian ini diharapkan menjadi suatu rekomendasi bagi pengajaran maupun penilaian berpikir kritis yang berkesinambungan supaya dapat diimplementasikan di kelas secara efektif. Kata Kunci: penilaian, berpikir kritis, menulis esei argumentative 3 Introduction Background of the study The requirement of critical thinking assessment is supporting by a general consensus that assessment is to appraise or estimate the level or magnitude of some attribute of a person (Mousavi, 2009: 36 in Brown and Abeywickrama, 2010: 3). However, many experts believe that it is still problematic; so it can not be viewed as an easy matter. The construction of critical thinking assessment should be in line with the principles of assessment in general: practicality, reliability, and validity. It has to take into account that the test is also used to measure someone’s ability (Brown & Abeywickrama, 2010: 4). Here, the ability of critical thinking is to be measured which relates to components or aspects of critical thinking as proposed by experts (Ennis, 1996: 4; Paul &Elder, 2007: 21; Toulmin, 2003: 11). These aspects of critical thinking that is common in a good writing are as follow: thesis, evidence, and reasoning and clarity. Similarly, the assessment should links to the learning objective of cognitive dimension (Anderson & Krathwohl, 2001). It is said that critical thinking relates to six educational objectives of Bloom’s Taxonomy (Brookhart, 2010: 84). Thus, in the sense of judgment, critical thinking is one kind of higher-order thinking. Brookhart says further that a good judgment should have a credible source. In order to know the students’ critical thinking, their judgment is to be assessed by evaluating the credibility of a source of information, identifying assumptions implicit in that information, and identifying rhetorical and persuasive methods. The explanation shows that critical thinking places the higher-order thinking. Although Brookhart has a more narrowed term, namely judgment; this notion is similar to general term of critical thinking. So, the present study will utilize Brookhart’s rubric to assess students’ critical thinking. There is assumption that a sustainable assessment of critical thinking is important to be conducted; it is due to several purposes (Ennis, 1993). First, it is to diagnose the students’ level of critical thinking prowess. Second, it is such an illuminating feedback so that the students are motivated to have a better critical thinking. Third, it informs teachers about the success of their teaching of critical thinking. Fourth, it provides a room for teachers to conduct a research dealing with critical thinking so that they know whether their students should enter educational program or not. Fifth, it gives information for holding schools accountable for the critical thinking prowess of their students. Sixth, in accordance with what has been claimed by Paul and Elder; it is to improve students’ ability to make their own thinking and decision through their discipline reasoning. These may become an illuminating assumption that scrutinizing critical thinking is very beneficial and it is recommended to administer a sustainable assessment. By looking back on Reichenbach’s concept which states that writing is as a mean to assess critical thinking, other experts claim that it is not only a standardized system for communication but also as an essential tool for learning (Alderson and Bachman, 2002: 4).This explanation correlates to the use of writing argumentative essay which can become a way for someone to reflect and facilitate her/ his critical thinking capacity. Moreover, argumentative essay is a reflection of critical thinking elements such as claim or thesis, reason and evidence, conclusion and clarity (Ennis 1996: 4; Emilia, 2011: 29); Paul & Elder, 2007: 21; Toulmin, 2003: 11). In order to write persuasively, the one should be able to distinguish between fact and opinion (McPherson, 1979: 193). Therefore, “conviction” and “persuasion” should be taken as a tool in making argument (Gardiner, 2004: 13). In other words, these elaborations tell us that a good argument should accommodate reasoning (contains fact, opinion, and persuasion) and conclusion, which links to the aspects of critical thinking to be used. Therefore, among these aspects of critical thinking, it is arguably to choose three that seems common to almost all good writing: thesis, evidence, and reasoning and clarity. Regarding to its administering, among other things, Brookhart’s Rubric can be used. Hence, it should be kept in mind that it is not the aspects of writing to be measured; but, critical thinking aspects are the main point. In order to have a fruitful result, this study is supported by previous studies (Fliegel & Holland, 2007 and Flores, 2007). The former, Facione Critical Thinking Scoring Rubric is used by Fliegel & Holand. The latter, Flores employed Toulmin’s 1958 model of argument and Knudson’s 1992 rubric. Method This study employed qualitative design since it involved purposive participants. They were six of third year students of a Private University in Ciamis who were selected based on their GPA (IPK/ Indeks Prestasi Kumulatif). To obtain the first data, task-based assessment was administered and students were asked to compose an English argumentative essay with a controversial issue: the corruptor should be 4 given a death penalty [my own emphasize]. Subsequently, these six essays were scored holistically by three raters who had a comprehensive knowledge both in critical thinking and writing. In order to avoid bias and the next scorer unable to see any previous score, not only were essays scored by three raters or scored at least three times, but also “mask” was used to conceal the significance of prior scores (Fliegel & Holland: 2007 and Brown & Abeywickrama, 2010 ). Subsequently, a self-administered questionnaire was conducted in the presence of the researcher so that the data were to be completed in one occasion rapidly. Findings and Discussions Description of Critical Thinking Assessment through Writing Argumentative Essay In scoring the essay, Brookhart’s Rubric was utilized which range of each aspects (thesis, evidence, and reasoning and clarity) are from 0 to 2. The results of scores were accumulated and the highest score was Student 1 (4.7), followed by Student 3 (4), Student 4 (3.3); and Student 2, Student 5, and Student 6 obtain the same score (3). To be more clearly, it was displayed in Figure 1 and Figure 2. The results which were presented in Figure 1 below indicated that students were able to produce judgment through argument. The judgments reflected their ability of critical thinking that basically had existed in their mind. However, their ability seemed to be hampered by language and writing skill. English language, which was considered as foreign language in Indonesia, affected students’ production of verbal communication. Regarding to writing skill, it was in line with Flores’s findings that students’ ability to transfer oral argument into written argument was to be constrained because the students’ ability depended on their awareness of the components of a good argument which was cued and prompted by the partner”. Overall, students’ ability in verbal communication needed improving . Figure 1. Result of Score for Critical Thinking Aspects Meanwhile, as given in Figure 2, the highest achiever (Student 1) held the first rank of score and gained the following score: thesis (1.7), evidence (1), and reasoning and clarity (2). This meant that Student 1 produced a clear and appropriate statement of assumption; mostly clear, relevant, and complete evidence but not accurate; and sound reasoning and clarity. This result was an expected result which corresponded to what was said by Wade & Tavris (2007: xxix); critical thinking was supported by intelligence. Also, this implied that Student 1 deserved the best student of all in critical thinking in this study. In addition, the result of study conducted by Flores (2007) could also be corresponded in term of writing skill. It was said that the students’ ability in writing may be influenced by her ability in transferring oral argument into written argument. The ability and achievement of Student 1 was an evidence of good critical thinker and she also considered as a good writer. Hence, an unexpected result emerged which was not relevant to Wade & Travis thought; the higher the student’s GPA the better their critical would be. In contrast, it was not as it was supposed to be. The result showed that the rank of score did not follow the rank-order of students’ GPA. Instead, as presented in Figure 2, the first mid achiever (Student 3) had a better critical thinking ability than the second high achiever (Student 2). The last three students (Student 2, Student 5, and Student 6) obtained score of 3 on average which meant that their verbal communication and writing skill needed improving. By looking back on Figure 1, it showed that the students were able to state all critical thinking aspects. So, it could be implied that they were indeed aware of critical thinking and they could express their own thoughts or judgment. Further, it depicted that their thesis’ score were only 1 on average. They stated an incomplete thesis; but, it was clear just for at the least partially answering the question posed by the problem or task. Similarly, the score of evidence was to be said that they were still poor in 5 accommodating accurate evidence. As stated previously above, they only stated their light opinions without supporting by any facts. They only stated evidence clearly, relevant, and complete. It could be observed through the Figure 1 that their score of evidence were only at 1 on average. Again, it could be said that not only were they hampered by the language they used but also their writing skill. The fact that they could reflect their critical thinking through their argumentative essay, this finding proved that they were able to analyze the phenomenon by making a judgment, and it followed by stating their own point of view to produce any recommendation (Reichenbach, 2001:19). Figure 2. Score Rank from Upper to Lower In relation to the finding above, there was a contributing factor regarding to the chosen topic. As the topic was chosen with purpose by researcher, it was believed that they were interested to this topic which was very controversial. It was believed that controversial issue can trigger students’ critical thinking. As it was understood, this was such a nation and public issue so that automatically they expressed the different point of view (Yuyun, 2010:110). Evidently, they succeed in persuading audiences by stating opinions which met to the experts (see Gardiner, 2004: 13 and Yuyun, 2010: 110). It could be concluded that the students’ performance of critical thinking which were reflected in their writing argumentative essays, were on the modest level; and of course they need improving. In conjunction to their level and the functions of critical thinking assessment (Ennis, 1993), as it was said previously that, among other things, was to know the level of their critical thinking. Also, this study accommodated another functions; it was a good feedback both for students and teachers. Finally, students were motivated to enhance their critical thinking ability for encountering the world which kept changing. Students’ Responses to Critical Thinking Assessment through Writing Argumentative Essay Regarding to second research question; what are students’ responses to critical thinking assessment through writing argumentative essay. This research question was accommodated by giving them two questions: what do you think about critical thinking assessment through writing argumentative essay and what are the purposes? Based on the result of questionnaires, students were aware of the beneficial value of critical thinking assessment. In addition, these responses were in accordance with Ennis’ purpose of critical thinking assessment (see Introduction). It was also in line with Reichenbach for its function of writing toward critical thinking; it was to assess critical thinking. Their responses could be taken into conclusion such as follow: (1) it was very beneficial since it provided feedback, self-assessment, and motivation for students to have a better critical thinking; (2) it could develop both critical thinking and writing skill; (3) it could stimulate and trigger students’ critical thinking ability; and (4) it was recommended by students that critical thinking assessment was to be conducted by institution. The result indicated that students wanted to be an agent of change and to survive in the competitive world. It had been explained the finding of the study. That was to say that the findings provided serious implication for the assessment, teaching and learning process. This holistic assessment which used Brookhart rubric had covered all stages of Taxonomy Bloom’s and it could not be scored in one occasion. It had to be interpreted by teachers as a self-reflection; explicit teaching of critical thinking in the classroom needed implementing effectively. Among other things, strategy of writing argumentative essay could be used because as previously discussed; it could facilitate students’ critical thinking. Conclusion This study suggests that the assessment does not rely on writing aspects, but it pay more attention to aspects of critical thinking which are common to a good writing: thesis, evidence, reasoning and clarity. The result indicates that students are able to utilize writing task to facilitate their critical thinking capacity. Since they were handicapped by their ability both in English language and writing skill, 6 students’ performances are still poor and their ability of verbal communication needs improving. Deal with students’ responses, the result of interview was in line with the experts’ assumptions on the function of critical thinking assessment: as feedback, self-assessment, motivation, and information for the teacher about the level of students’ critical thinking. Therefore, it is recommended that it should be implemented in the classroom. Hopefully, this study can become an illuminating resource for other research. References Alderson, J. Charles & Bachman, Lyle F. 2002. Assesing Writing. Cambridge: Cambridge University Press. Anderson, L. W., & Krathwohl, D. R. (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. New York: Longman. Brookhart, Susan M. 2010. How to Assess Higher Order Thinking Skills in Your Classroom. Alexandria: ASCD Brown, Douglas H. & Abeywickrama, Priyanada. 2010. Language Assessment- Principles and Classroom Practices. Second Edtion. New York: Pearson Longman. Emilia, Emi. 2011. Teaching Academic Writing- A Critical Genre- Based Approach in an EFL Context. Saarbrücken: Lambert Academic Pubishing. Ennis, H. Robert. 1993. Critical Thinking Assessment. Theory into Practice, Volume 32, Number 3, Summer 1993. Copyright 1993 College of Education, The Ohio State University.(http://www3.qcc.cuny.edu/WikiFiles/file/Ennis%20Critical%20Thinking%20Assess ment.pdf, date of access: 24th November 2013). Ennis, Robert H. 1996. Critical Thinking. New Jersey: Prentice Hall. Fliegel, Richard & Holland, John. 2007. Assessing Critical Thinking through Students’ Writing. (http://cet.usc.edu/resources/teaching_learning/material_docs/John_Holland__Richard_Fliegels_ presentation.ppt., date of access: 23rd November 2013). Flores, Eden Regala. 2007. Thinking Skills Reflected in the Argumentative Essays of Freshman College Students: A Descriptive Analysis. The Asia Pacific-Education Researcher Vol. 16 No. 1, P. 3344. Retrieved from http://www.dlsu.edu.ph/research/journals/taper/pdf/200706/Flores.pdf, date of accessed: 11TH July 2013. Gardiner, J. H. 2004. The Making of Argument.Retrieved from http://booksiread.org, date of accessed: 3rd August 2004. McPherson, Elisabeth. 1979. Language Arts in the Community College: A Process Model in Three Language-Arts Curriculum Models- Pre Kindrgarten through College, p. 188-202. Illinois: National Council. Paul, Richard & Elder, Linda. 2007. Consequential Validity: Using Assessment to Drive Instruction. Foundation for Critical Thinking, page 2. (http://www.criticalthinking.org/files/White%20PaperAssessmentSept2007.pdf. ) Reichenbach, Bruce R. 2001. Introduction to Critical Thinking. New York: McGraw Hill. Toulmin, Stephen. 2003. The Uses of Argument. New York: Cambridge University Press. Wade, Carole & Tavris, Carol. 2007. Psikologi. Edisi Kesembilan Yuyun, Ignasia. 2010. A Study of Arguments in Senior High School Debate. Jurnal 14, No. 1, November 2010, p.109-133. Penelitian Vol. 7 Appendices Appendix 1: Writing Prompt Write an argumentative essay on the issue :The corruptor should be given a death penalty Appendix 2: Students’ Argumentative Writing 8 9 10 11 12 13 Appendix 3: Answers of Questionnaire 14 15 16 Appendix 4: Translated Answers of Questionnaire Students High achiever Result of interview Std. 1 I completely agreed to organize critical thinking assessment through writing Argumentative essay. It can determine someone’s efficacy to be a better person. Std. 2 I thinks “critical thinking assessment through writing Argumentative essay” is a good idea. It provides students’ information about their knowledge, recent issues, and their efficacy. It also provides feedback and self assessment for the students. It is as a device to solve the problem. Middle achiever Std. 3 “Critical thinking assessment through writing Argumentative essay” is an effective activity to express students’ idea/opinion. Further, Argumentative essay can develop and increase students’ critical thinking. The purpose of assessment is to measure students’ critical thinking and to think actively. Std. 4 I think “Critical thinking assessment through writing Argumentative essay” is a good idea. It develops not only critical thinking but also writing ability. In addition, it helps students to solve the problem. Low achiever Std. 5 I think “Critical thinking assessment through writing Argumentative essay” is appropriate in developing writing skill. It should be organized because it is beneficial for the life. Std. 6 It is beneficial to administer because it can stimulate students’ critical thinking.