assessment- full paper -1st revised.doc

1
Assessment of Critical Thinking through Writing an English Argumentative
Essay
A Research-Based Paper
Submitted to the School of Postgraduate Studies
English Department Program
in Partial Fulfillment of the Requirements for the Final Test of Language Testing and
Assessment
Disusun Oleh :
DEDEH ROHAYATI
STUDENT NUMBER: 1201025
CLASS 3A
ENGLISH EDUCATION DEPARTMENT
SCHOOL OF POSTGRADUATE STUDIES
INDONESIA UNIVERSITY OF EDUCATION
2
Assessment of Critical Thinking through Writing an English Argumentative Essay
Dedeh Rohayati
Indonesia University of Education
rohayatidedeh@rocketmail.com
Abstract
In connection with critical thinking assessment which is still problematic, many experts propose
various techniques to accommodate it. They believe that among other things, writing
argumentative essay can be applied to assess critical thinking ability. Thus, the use of Brookhart’s
rubric seems to be applicable to assess three aspects of critical thinking such as thesis, evidence,
and reasoning and clarity. Supported by these notions, this study is therefore aimed to report how
to assess critical thinking through writing argumentative essay and to investigate the students’
responses. By utilizing qualitative design and a simple descriptive statistic, six students’ English
argumentative essays are collected. The result shows that the students are able to state these
aspects; yet, they are poor in expressing accurate evidence of the phenomenon. Overall, their
critical thinking ability needs improving. On the other hand, based on questionnaires answers,
this study, to a great extent, reveals that critical thinking assessment is beneficial to be conducted
because it is considered as a feedback for student. In addition, it can motivate them to have a
better critical thinking. To be concluded, this study appears to be good evidence that critical
thinking assessment can be administered through writing. Evidently, it is recommended that both
sustainable teaching critical thinking and its assessment should be implemented in the classroom
effectively.
Key words: assessment, critical thinking, writing argumentative essay
Abstrak
Berkaitan dengan penilaian berpikir kritis yang masih problematik, para ahli di bidangnya
mengusulkan berbagai tekhnik diantaranya melalui tulisan argumentative esei. Sehingga,
penggunnaan rubric Brookhart nampaknya dapat diaplikasikan untuk menilai aspek-aspek
berpikir kritis seperti tesis (thesis), pembuktian (evidence), alasan (reason) dan kejelasan
(clarity). Pemikiran tersebut telah mendorong penelitian ini dilakukan dengan tujuan untuk
memaparkan penilaian berpikir kritis melalui tulisan esei argumentative sekaligus untuk
mengetahui responnya dari para mahasiswa. Penelitian kualitative yang menggunakan statistik
sederhana ini, menggunakan 6 argumentative esei bahasa Inggris. Hasil penelitian menunjukan
bahwa mereka mampu menyatakan semua aspek-aspek berpikir kritis tersebut, namun belum
maksimal dalam mengungkapkan pembuktian-pembuktian yang akurat. Sedangkan dari hasil
jawaban kuesioner, secara garis besar mereka merasakan pentingnya penilaian ini sebagai umpan
balik yang dapat memotivasi mereka untuk memiliki kemampuan berpikir kritis yang lebih baik.
Kesimpulannya, hasil penelitian ini menjadi suatu bukti bahwa penilaian berpikir kritis dapat
dilakukan berdasarkan hasil tulisan..Terakhir, hasil kajian ini diharapkan menjadi suatu
rekomendasi bagi pengajaran maupun penilaian berpikir kritis yang berkesinambungan supaya
dapat diimplementasikan di kelas secara efektif.
Kata Kunci: penilaian, berpikir kritis, menulis esei argumentative
3
Introduction
Background of the study
The requirement of critical thinking assessment is supporting by a general consensus that
assessment is to appraise or estimate the level or magnitude of some attribute of a person (Mousavi, 2009:
36 in Brown and Abeywickrama, 2010: 3). However, many experts believe that it is still problematic; so
it can not be viewed as an easy matter.
The construction of critical thinking assessment should be in line with the principles of
assessment in general: practicality, reliability, and validity. It has to take into account that the test is also
used to measure someone’s ability (Brown & Abeywickrama, 2010: 4). Here, the ability of critical
thinking is to be measured which relates to components or aspects of critical thinking as proposed by
experts (Ennis, 1996: 4; Paul &Elder, 2007: 21; Toulmin, 2003: 11). These aspects of critical thinking
that is common in a good writing are as follow: thesis, evidence, and reasoning and clarity. Similarly, the
assessment should links to the learning objective of cognitive dimension (Anderson & Krathwohl, 2001).
It is said that critical thinking relates to six educational objectives of Bloom’s Taxonomy
(Brookhart, 2010: 84). Thus, in the sense of judgment, critical thinking is one kind of higher-order
thinking. Brookhart says further that a good judgment should have a credible source. In order to know the
students’ critical thinking, their judgment is to be assessed by evaluating the credibility of a source of
information, identifying assumptions implicit in that information, and identifying rhetorical and
persuasive methods. The explanation shows that critical thinking places the higher-order thinking.
Although Brookhart has a more narrowed term, namely judgment; this notion is similar to general term of
critical thinking. So, the present study will utilize Brookhart’s rubric to assess students’ critical thinking.
There is assumption that a sustainable assessment of critical thinking is important to be
conducted; it is due to several purposes (Ennis, 1993). First, it is to diagnose the students’ level of critical
thinking prowess. Second, it is such an illuminating feedback so that the students are motivated to have a
better critical thinking. Third, it informs teachers about the success of their teaching of critical thinking.
Fourth, it provides a room for teachers to conduct a research dealing with critical thinking so that they
know whether their students should enter educational program or not. Fifth, it gives information for
holding schools accountable for the critical thinking prowess of their students. Sixth, in accordance with
what has been claimed by Paul and Elder; it is to improve students’ ability to make their own thinking and
decision through their discipline reasoning. These may become an illuminating assumption that
scrutinizing critical thinking is very beneficial and it is recommended to administer a sustainable
assessment.
By looking back on Reichenbach’s concept which states that writing is as a mean to assess
critical thinking, other experts claim that it is not only a standardized system for communication but also
as an essential tool for learning (Alderson and Bachman, 2002: 4).This explanation correlates to the use
of writing argumentative essay which can become a way for someone to reflect and facilitate her/ his
critical thinking capacity. Moreover, argumentative essay is a reflection of critical thinking elements such
as claim or thesis, reason and evidence, conclusion and clarity (Ennis 1996: 4; Emilia, 2011: 29); Paul &
Elder, 2007: 21; Toulmin, 2003: 11). In order to write persuasively, the one should be able to distinguish
between fact and opinion (McPherson, 1979: 193). Therefore, “conviction” and “persuasion” should be
taken as a tool in making argument (Gardiner, 2004: 13). In other words, these elaborations tell us that a
good argument should accommodate reasoning (contains fact, opinion, and persuasion) and conclusion,
which links to the aspects of critical thinking to be used. Therefore, among these aspects of critical
thinking, it is arguably to choose three that seems common to almost all good writing: thesis, evidence,
and reasoning and clarity. Regarding to its administering, among other things, Brookhart’s Rubric can be
used. Hence, it should be kept in mind that it is not the aspects of writing to be measured; but, critical
thinking aspects are the main point.
In order to have a fruitful result, this study is supported by previous studies (Fliegel & Holland,
2007 and Flores, 2007). The former, Facione Critical Thinking Scoring Rubric is used by Fliegel &
Holand. The latter, Flores employed Toulmin’s 1958 model of argument and Knudson’s 1992 rubric.
Method
This study employed qualitative design since it involved purposive participants. They were six of
third year students of a Private University in Ciamis who were selected based on their GPA (IPK/ Indeks
Prestasi Kumulatif). To obtain the first data, task-based assessment was administered and students were
asked to compose an English argumentative essay with a controversial issue: the corruptor should be
4
given a death penalty [my own emphasize]. Subsequently, these six essays were scored holistically by
three raters who had a comprehensive knowledge both in critical thinking and writing. In order to avoid
bias and the next scorer unable to see any previous score, not only were essays scored by three raters or
scored at least three times, but also “mask” was used to conceal the significance of prior scores (Fliegel &
Holland: 2007 and Brown & Abeywickrama, 2010 ). Subsequently, a self-administered questionnaire was
conducted in the presence of the researcher so that the data were to be completed in one occasion rapidly.
Findings and Discussions
Description of Critical Thinking Assessment through Writing Argumentative Essay
In scoring the essay, Brookhart’s Rubric was utilized which range of each aspects (thesis, evidence,
and reasoning and clarity) are from 0 to 2. The results of scores were accumulated and the highest score
was Student 1 (4.7), followed by Student 3 (4), Student 4 (3.3); and Student 2, Student 5, and Student 6
obtain the same score (3). To be more clearly, it was displayed in Figure 1 and Figure 2.
The results which were presented in Figure 1 below indicated that students were able to produce
judgment through argument. The judgments reflected their ability of critical thinking that basically had
existed in their mind. However, their ability seemed to be hampered by language and writing skill.
English language, which was considered as foreign language in Indonesia, affected students’ production
of verbal communication. Regarding to writing skill, it was in line with Flores’s findings that students’
ability to transfer oral argument into written argument was to be constrained because the students’ ability
depended on their awareness of the components of a good argument which was cued and prompted by the
partner”. Overall, students’ ability in verbal communication needed improving .
Figure 1. Result of Score for Critical Thinking Aspects
Meanwhile, as given in Figure 2, the highest achiever (Student 1) held the first rank of score and
gained the following score: thesis (1.7), evidence (1), and reasoning and clarity (2). This meant that
Student 1 produced a clear and appropriate statement of assumption; mostly clear, relevant, and complete
evidence but not accurate; and sound reasoning and clarity. This result was an expected result which
corresponded to what was said by Wade & Tavris (2007: xxix); critical thinking was supported by
intelligence. Also, this implied that Student 1 deserved the best student of all in critical thinking in this
study. In addition, the result of study conducted by Flores (2007) could also be corresponded in term of
writing skill. It was said that the students’ ability in writing may be influenced by her ability in
transferring oral argument into written argument. The ability and achievement of Student 1 was an
evidence of good critical thinker and she also considered as a good writer.
Hence, an unexpected result emerged which was not relevant to Wade & Travis thought; the higher
the student’s GPA the better their critical would be. In contrast, it was not as it was supposed to be. The
result showed that the rank of score did not follow the rank-order of students’ GPA. Instead, as presented
in Figure 2, the first mid achiever (Student 3) had a better critical thinking ability than the second high
achiever (Student 2). The last three students (Student 2, Student 5, and Student 6) obtained score of 3 on
average which meant that their verbal communication and writing skill needed improving.
By looking back on Figure 1, it showed that the students were able to state all critical thinking
aspects. So, it could be implied that they were indeed aware of critical thinking and they could express
their own thoughts or judgment. Further, it depicted that their thesis’ score were only 1 on average. They
stated an incomplete thesis; but, it was clear just for at the least partially answering the question posed by
the problem or task. Similarly, the score of evidence was to be said that they were still poor in
5
accommodating accurate evidence. As stated previously above, they only stated their light opinions
without supporting by any facts. They only stated evidence clearly, relevant, and complete. It could be
observed through the Figure 1 that their score of evidence were only at 1 on average. Again, it could be
said that not only were they hampered by the language they used but also their writing skill. The fact that
they could reflect their critical thinking through their argumentative essay, this finding proved that they
were able to analyze the phenomenon by making a judgment, and it followed by stating their own point of
view to produce any recommendation (Reichenbach, 2001:19).
Figure 2. Score Rank from Upper to Lower
In relation to the finding above, there was a contributing factor regarding to the chosen topic. As
the topic was chosen with purpose by researcher, it was believed that they were interested to this topic
which was very controversial. It was believed that controversial issue can trigger students’ critical
thinking. As it was understood, this was such a nation and public issue so that automatically they
expressed the different point of view (Yuyun, 2010:110). Evidently, they succeed in persuading audiences
by stating opinions which met to the experts (see Gardiner, 2004: 13 and Yuyun, 2010: 110).
It could be concluded that the students’ performance of critical thinking which were reflected in
their writing argumentative essays, were on the modest level; and of course they need improving. In
conjunction to their level and the functions of critical thinking assessment (Ennis, 1993), as it was said
previously that, among other things, was to know the level of their critical thinking. Also, this study
accommodated another functions; it was a good feedback both for students and teachers. Finally, students
were motivated to enhance their critical thinking ability for encountering the world which kept changing.
Students’ Responses to Critical Thinking Assessment through Writing Argumentative Essay
Regarding to second research question; what are students’ responses to critical thinking assessment
through writing argumentative essay. This research question was accommodated by giving them two
questions: what do you think about critical thinking assessment through writing argumentative essay and
what are the purposes? Based on the result of questionnaires, students were aware of the beneficial value
of critical thinking assessment. In addition, these responses were in accordance with Ennis’ purpose of
critical thinking assessment (see Introduction). It was also in line with Reichenbach for its function of
writing toward critical thinking; it was to assess critical thinking. Their responses could be taken into
conclusion such as follow: (1) it was very beneficial since it provided feedback, self-assessment, and
motivation for students to have a better critical thinking; (2) it could develop both critical thinking and
writing skill; (3) it could stimulate and trigger students’ critical thinking ability; and (4) it was
recommended by students that critical thinking assessment was to be conducted by institution. The result
indicated that students wanted to be an agent of change and to survive in the competitive world.
It had been explained the finding of the study. That was to say that the findings provided serious
implication for the assessment, teaching and learning process. This holistic assessment which used
Brookhart rubric had covered all stages of Taxonomy Bloom’s and it could not be scored in one occasion.
It had to be interpreted by teachers as a self-reflection; explicit teaching of critical thinking in the
classroom needed implementing effectively. Among other things, strategy of writing argumentative essay
could be used because as previously discussed; it could facilitate students’ critical thinking.
Conclusion
This study suggests that the assessment does not rely on writing aspects, but it pay more attention
to aspects of critical thinking which are common to a good writing: thesis, evidence, reasoning and
clarity. The result indicates that students are able to utilize writing task to facilitate their critical thinking
capacity. Since they were handicapped by their ability both in English language and writing skill,
6
students’ performances are still poor and their ability of verbal communication needs improving. Deal
with students’ responses, the result of interview was in line with the experts’ assumptions on the function
of critical thinking assessment: as feedback, self-assessment, motivation, and information for the teacher
about the level of students’ critical thinking. Therefore, it is recommended that it should be implemented
in the classroom. Hopefully, this study can become an illuminating resource for other research.
References
Alderson, J. Charles & Bachman, Lyle F. 2002. Assesing Writing. Cambridge:
Cambridge University Press.
Anderson, L. W., & Krathwohl, D. R. (2001). A Taxonomy for Learning, Teaching, and Assessing: A
Revision of Bloom’s Taxonomy of Educational Objectives. New York: Longman.
Brookhart, Susan M. 2010. How to Assess Higher Order Thinking Skills in Your Classroom. Alexandria:
ASCD
Brown, Douglas H. & Abeywickrama, Priyanada. 2010. Language Assessment- Principles and
Classroom Practices. Second Edtion. New York: Pearson Longman.
Emilia, Emi. 2011. Teaching Academic Writing- A Critical Genre- Based Approach in an EFL
Context. Saarbrücken: Lambert Academic Pubishing.
Ennis, H. Robert. 1993. Critical Thinking Assessment. Theory into Practice, Volume 32,
Number 3, Summer 1993. Copyright 1993 College of Education, The Ohio State
University.(http://www3.qcc.cuny.edu/WikiFiles/file/Ennis%20Critical%20Thinking%20Assess
ment.pdf, date of access: 24th November 2013).
Ennis, Robert H. 1996. Critical Thinking. New Jersey: Prentice Hall.
Fliegel, Richard & Holland, John. 2007. Assessing Critical Thinking through Students’ Writing.
(http://cet.usc.edu/resources/teaching_learning/material_docs/John_Holland__Richard_Fliegels_
presentation.ppt., date of access: 23rd November 2013).
Flores, Eden Regala. 2007. Thinking Skills Reflected in the Argumentative Essays of Freshman College
Students: A Descriptive Analysis. The Asia Pacific-Education Researcher Vol. 16 No. 1, P. 3344. Retrieved from http://www.dlsu.edu.ph/research/journals/taper/pdf/200706/Flores.pdf, date of
accessed: 11TH July 2013.
Gardiner, J. H. 2004. The Making of Argument.Retrieved from
http://booksiread.org, date of
accessed: 3rd August 2004.
McPherson, Elisabeth. 1979. Language Arts in the Community College: A Process Model in Three
Language-Arts Curriculum Models- Pre Kindrgarten through College, p. 188-202. Illinois:
National Council.
Paul, Richard & Elder, Linda. 2007. Consequential Validity: Using Assessment to Drive
Instruction. Foundation for Critical Thinking, page 2.
(http://www.criticalthinking.org/files/White%20PaperAssessmentSept2007.pdf. )
Reichenbach, Bruce R. 2001. Introduction to Critical Thinking. New York: McGraw Hill.
Toulmin, Stephen. 2003. The Uses of Argument. New York: Cambridge University Press.
Wade, Carole & Tavris, Carol. 2007. Psikologi. Edisi Kesembilan
Yuyun, Ignasia. 2010. A Study of Arguments in Senior High School Debate. Jurnal
14, No. 1, November 2010, p.109-133.
Penelitian Vol.
7
Appendices
Appendix 1: Writing Prompt
Write an argumentative essay on the issue :The corruptor should be given a death penalty
Appendix 2: Students’ Argumentative Writing
8
9
10
11
12
13
Appendix 3: Answers of Questionnaire
14
15
16
Appendix 4: Translated Answers of Questionnaire
Students
High achiever
Result of interview
Std. 1
I completely agreed to organize critical thinking assessment through writing
Argumentative essay. It can determine someone’s efficacy to be a better
person.
Std. 2
I thinks “critical thinking assessment through writing Argumentative essay”
is a good idea. It provides students’ information about their knowledge, recent
issues, and their efficacy. It also provides feedback and self assessment for the
students. It is as a device to solve the problem.
Middle achiever
Std. 3
“Critical thinking assessment through writing Argumentative essay” is an
effective activity to express students’ idea/opinion. Further, Argumentative
essay can develop and increase students’ critical thinking. The purpose of
assessment is to measure students’ critical thinking and to think actively.
Std. 4
I think “Critical thinking assessment through writing Argumentative essay” is
a good idea. It develops not only critical thinking but also writing ability. In
addition, it helps students to solve the problem.
Low achiever
Std. 5
I think “Critical thinking assessment through writing Argumentative essay” is
appropriate in developing writing skill. It should be organized because it is
beneficial for the life.
Std. 6
It is beneficial to administer because it can stimulate students’ critical
thinking.