Thinking Skills and Creativity 29 (2018) 203–212
Contents lists available at ScienceDirect
Thinking Skills and Creativity
journal homepage: www.elsevier.com/locate/tsc
Fostering the skills of critical thinking and question-posing in a
project-based learning environment
T
⁎
Irit Sassona,b,c, , Itamar Yehudaa,b, Noam Malkinsonb
a
b
c
Tel-Hai College, Israel
Shamir Research Institute, Israel
University of Haifa, Israel
A R T IC LE I N F O
ABS TRA CT
Keywords:
High-order thinking skills
Critical thinking
Question-posing skill
Educational effectiveness research
Constructivism
Innovation in education
Innovative pedagogical models for teaching and learning aimed at developing higher order
thinking skills require more sophisticated evaluation mechanisms than traditional pedagogical
models to determine their effectiveness. In recent years, increased implementation of creative
pedagogy has stimulated a parallel interest in the field of educational effectiveness research
(EER). EER studies the factors impacting educational outcomes. This research examined an innovative program for 9th and 10th graders. The program implemented a project-based learning,
constructivist approach with three teachers co-teaching each lesson to maximize development of
high-order thinking skills. Students learned the required ministry of education material for all
subjects through projects based on group work and peer learning. The research goal was to
evaluate the innovative program’s effect on two skills: critical thinking and question-posing. The
innovative class was compared to a traditional class learning the same material at three points in
time over two years using pre- and post-case-based questionnaires (71 students, total of 192
questionnaires). Although no significant differences were found between the classes in the critical
thinking pre-questionnaire, students in the innovative learning environment had a significant
advantage in this skill after two years. Significant differences in question-posing were found in
the pre-questionnaire and the gaps enlarged over the research period. The results emphasize the
importance and contribution of a case-based evaluation method for "evidence-based education."
1. Introduction
The education agenda of schools worldwide is much broader than teaching elementary reading, writing and arithmetic abilities; it
is education for thinking (Zohar & David, 2009). High-order thinking (HOT) skills have been described as complex skills with no
simple algorithm for problem-solving. The patterns of thought and action involved are not obvious or predefined, and occasionally
culminate in a multitude of answers, each one having its own advantages and disadvantages (Resnick, 1987). Skills of high-order
thinking include Bloom's Taxonomy levels of analysis, synthesis, evaluation, comprehension and application as well as problem
solving and critical thinking (Ennis, 1985; Tsaparlis & Papaphotis, 2009). HOT skills are the focus of many different educational
programs. There are different approaches as of what are the best programs or models for developing thinking skills, but there is
unanimity within all who engage in the field of education, that the primary goal is - education for thinking (Zohar & David, 2009).
Using the term high-order thinking indicates there is also a low-order thinking. As Zoller and Tsaparlis (1997) explain, low-order
⁎
Corresponding author at: Tel-Hai College, Israel.
E-mail address: iritsa@telhai.ac.il (I. Sasson).
https://doi.org/10.1016/j.tsc.2018.08.001
Received 1 April 2018; Received in revised form 1 August 2018; Accepted 4 August 2018
Available online 07 August 2018
1871-1871/ © 2018 Published by Elsevier Ltd.
Thinking Skills and Creativity 29 (2018) 203–212
I. Sasson et al.
thinking is based on simple recall and application of information that is well familiar to the learner from past experiences. While loworder thinking is the application of repeated past experiences, high-order thinking is the individual's interpretation, analyzation and
manipulation of it. Therefore, low and high-order thinking vary individually (Lewis & Smith, 1993). High-order thinking skills
generate new knowledge by exercising judgment, critique and creativity and are relevant to life outside of the classroom (Zohar &
David, 2009). One of the primary goals of education is to prepare students to join the workforce with analytical, problem-solving and
critical thinking abilities so that they can perform at a higher level of thinking (Brierton, Wilson, Kistler, Flowers, & Jones, 2016).
Students who acquire good HOT skills become productive members of workforces that produce knowledge, promote information
sharing and foster progress that help build prosperous and cohesive societies (Costa, 2008).
Two primary components of HOT skills curricula are the process of constructing knowledge and developing learner curiosity.
Instructional methods emphasize coaching and guiding students in the use of various resources and employing clear instructions in a
scaffolding learning structure (King, Goodson, & Rohani, 1998; Swartz, 2008). The constructivism approach and enhancing question
posing among students are also highly supportive of developing HOT skills, curiosity and innovative thinking (Craft & Chappell,
2016; David, 2008; Dori & Herscovitz, 1999; Dori & Sasson, 2008).
The goal of this research was to evaluate the effect of a new pedagogical approach on two key HOT skills: critical thinking and
question-posing. The innovative program for 9th and 10th graders applied the constructivism approach using project-based learning
with three teachers co-teaching each lesson to maximize development of high-order thinking skills. It is an Educational Effectiveness
Research (EER) type, which focuses on the study of the factors that may have an impact on educational outcomes, within innovative
pedagogical models (Reynolds et al., 2014). This type of research can provide feedback on how to improve school performance and
on the inculcation of the values and attitudes important to policy makers and educators (Evans, Zeun, & Stanier, 2014; Roschelle,
Penuel, Yarnall, Shechtman, & Tatar, 2005). In addition it is an important tool in professional development, evaluating teacher
effectiveness and the outcomes of different teaching methods (Tigelaar & Beijaard, 2013; Trevisan, 2007). The new pedagogical
programs being implemented worldwide require a tool that can assess learning aspects such as critical thinking and other HOT skills
difficult to measure in order to collect information about program progress (Sharples et al., 2015; Trevisan, 2007). Yet, much research
is still needed in order to identify the measures of learning processes and their outcomes and in extracting useful data from them
(Roschelle et al., 2005; Sharples et al., 2015).
Conservative teaching restricted information and ideas to those found in texts. As students are exposed to today's internet and
technology, they become exposed to unlimited information, ideas and tools to support and enhance their learning (Windschitl, 2002).
But how can curriculum direct students to obtain successful HOT skills? There is a gap between teachers' education and their ability
to teach HOT skills in the classroom (Frid, 2000). Practicing constructivism is complex, mainly due to lack of understanding its
broader concept (Windschitl, 2002). Even teachers with a constructivist-oriented outlook do not necessarily teach actively. The
literature indicates several reasons for inconsistencies between teacher perceptions and instructional practices. Lack of sufficient class
time and the need for specialized professional training are prominent (Ertmer, 2005; Liu, 2011; Sandholtz & Reilly, 2004). The
complexity of designing curriculum; mastering skills and strategies for successful facilitation; honoring the diversity of students'
beliefs and backgrounds are just a few of the skills teachers must acquire to practice constructivism in the classroom (Windschitl,
2002). In reality, inside the classroom, teachers find it extremely difficult to implement constructivist models in stimulating ways
(David, 2008). Success in applying constructivist approaches depends upon the teacher's ability to underpin its theoretical concept;
otherwise, their teaching remains superficial. Assessment of the pedagogical quality of educational innovations is necessary to learn
about their application and educational potential.
The possible gap between theory and practice and the need for assessment tools to assess HOT skills, presented above, constitute
the significant of this study which examined an innovative program characterized by the constructivism approach. In order to assess
the students' high-order thinking skills and thus the effectiveness of the program, a pre-and post-case-based questionnaire was
developed and validated.
2. Literature review
The literature review that follows describes current research on the constructivist approach, in particular on project-based
learning (PBL), critical thinking, and question-posing.
2.1. Constructivism and PBL
According to constructivism, students independently and actively build their cognitive world by mutual interaction with their
surroundings during real learning events. Constructivist learning environments expose their learners to a variety of information
sources, allow the processing of information to knowledge from different points of view, create a connection between the learning
process and the learners’ personal experience and culture, support the use of diverse learning styles and encourage reflection on the
learning process. Knowledge is the product of active building and reflects the image of reality constructed by the individual. This
process, despite its personal and subjective nature, is greatly affected by social interaction (Atkins, 1993; Ertmer & Newby, 1993;
Nagowah & Nagowah, 2009; Rivet & Krajcik, 2004; Rosenfeld & Rosenfeld, 2006; Von Glaserfeld, 1991).
Researchers have found that constructivist theory has the potential to improve educational outcomes for students in higher
education (Lea, Stephenson, & Troy, 2003; Loyens & Gijbels, 2008). These findings are generating educational reforms promoting
implementation of constructivism models in innovative pedagogical programs in secondary schools and educational centers (Frid,
2000). In these programs, educational leaders embrace the concept of constructing knowledge based on students' own learning
204
Thinking Skills and Creativity 29 (2018) 203–212
I. Sasson et al.
experiences (Bednar, Cunningham, Duffy, & Perry, 1992; Bruning, Schraw, & Ronning, 1999; Loyens & Gijbels, 2008). Such learning
develops deep understanding. Students are engaged in collaborative work, develop HOT skills such as reasoning and problem solving
and are able to transfer this knowledge to new situations with meaningful contexts (Dori & Sasson, 2013; Mansilla & Gardner, 2008;
Perkins, 1986; Sasson & Dori, 2015). Such skills are the essential foundation for adaptation to the everyday personal, social and
professional demands of the 21st century (Becker, 2000).
Different teaching methods which apply the constructivist approach have been found effective in developing critical thinking and
question-posing skills. Holmes, Wieman, and Bonn, (2015) found that explicit instruction given to introductory physics laboratory
course students for the acquisition of data and its relation to scientific models positively affects the development of critical thinking.
Students in the experimental group demonstrated greater critical thinking than control group students. Based on Bloom's taxonomy,
high critical thinking was expressed when students supported data through arguments and integrated reflection during scientific
analysis. Dori, Tal, and Tsaushu, (2003) found that teaching biotechnology through case studies positively affects the development of
students' ability in argumentation, question-posing, and system thinking. The method was found effective for all students at all
academic levels. Kaberman and Dori (2009) investigated the effect of metacognitive training on question-posing skill in chemistry
studies. Students in the experimental group significantly improved their ability to ask questions, reflected in an increase in the level of
thinking of the question and in the depth of chemistry understanding.
Problem-Based Learning (PBL) is an effective constructivist model for developing HOT skills. PBL challenges students to actively
design and resolve questions or problems (scientific investigation) through realistic, real-world projects that are interesting and
provoke thinking (David, 2008; Epstein, 2008; Thomas, 2000). This model exposes students to decision making, problem solving and
different types of investigative activities. PBL develops thinking skills, social skills and the ability to collaborate effectively.
2.2. Critical thinking and question-posing
In-depth knowledge leads to the ability to think critically and to formulate reasoned stances and viewpoints (Zohar & David,
2009). Critical thinking requires students to be thorough, purposeful and deliberate, to focus on the issue at hand, fully evaluate all
parts of its complex, challenging claims and arguments. Therefore, a thinking process which includes critical thinking, problem
solving, creative thinking and decision making based upon all interpreted information – enhances high-order thinking (Lewis &
Smith, 1993; Madhuri, Kantamreddi, & Prakash Goteti, 2012). Two key aspects of critical thinking are planning and reflecting,
therefore hands-on activities promote these aspects especially when practiced as a regular part of the day (Ennis, 1985; Epstein,
2008). Perkins and Ritchhart (2004)) emphasized the importance of developing critical thinking for use in daily life. Young people
and adults should think actively when confronted with an argument or unreasonable rumor, when problem solving, or when listening
to sweeping political statements. Humans are constantly thinking and often self-aware; however, studies show that people who do not
pay attention to their thinking process miss many opportunities for creative critical thinking. Good critical thinkers make better
judgments and decisions in complex situations (Gambrill, 2006); they are more likely to reach better results and become more active
citizens (Barton & McCully, 2007; Holmes & Clizbe, 1997). Since solving real world problems requires application of critical thinking,
acquiring its skills prepares students for the world outside of school in the workplace and in interpersonal and social contexts where
decisions are to be made carefully and independently on a daily basis (Becker, 2000; Brierton et al., 2016).
Critical thinking involves a variety of skills such as analysis, evaluation and synthesis (Dwyer, Hogan, & Stewart, 2011). Analysis
is a critical thinking skill used to detect, examine and identify the propositions within an argument and the role they play. Evaluation
is used in assessing propositions and the conclusions they infer with respect to their reliability, relevance, logical strength and the
potential for omissions, bias and imbalance in the argument; thus, deciding the overall strength or weakness of the argument.
Synthesis includes the collection of reliable, relevant and logical evidence based on the previous analysis and evaluation of existing
evidence for the purpose of drawing a reasonable conclusion (Facione, 1990). In order for critical thinking to develop to an optimal
level, related metacognitive processes may be needed to support both critical thinking skill development and the successful application of critical thinking to real-world problems. Reflective thinking is one such metacognitive process whose function is to raise
doubts about beliefs, perceptions and methods of action (Ennis, 1989). Planning and reflecting processes are most effectively developed through hands-on activities especially when practiced as a regular part of the day (Ennis, 1985; Epstein, 2008).
Question-posing is an essential educational tool in the teaching and learning of all disciplines (Dori & Herscovitz, 1999). The
ability to pose complex questions and analyze them when answers cannot be found in the text requires a metacognitive function
within a thought process (Dori & Sasson, 2008; Kaberman & Dori, 2009). Such thinking skill is a unique feature of HOT that enables
students to re-apply the same thinking process to future tasks (Lee, 1980; Sasson & Dori, 2015; Swartz, 2008).To encourage HOT, the
questions should be aimed to deepen comprehension by practicing and improving the ability to demonstrate, compare, argue and
justify arguments, solve problems, formulate hypotheses, offer explanations and assess understanding. Such questions do not merely
refer to knowledge, but rather ask students to act upon knowledge (Karmon, 2007; Resnick, Michaels, & O’Connor, 2010). While an
effective strategy for improving inquiry and problem-solving ability is to foster students’ question-posing and critical thinking skills
(Cheng & Wan, 2017; Kaberman & Dori, 2009), there are many challenges in implementing innovative constructivist teaching and
learning methods that emphasize the development of high-order thinking.
3. Methodology
The goal of the research was to evaluate the effectiveness of an innovative high school pedagogical model characterized by
constructivism. The model employed a comprehensive approach to teaching the 9th and10th grade curricula. Students learned the
205
Thinking Skills and Creativity 29 (2018) 203–212
I. Sasson et al.
required ministry of education material for all subjects through projects based on group work and peer learning. The objectives of the
program were developing independent learning skills, encouraging inquiry and creativity, and learning teamwork. There are about
50 students in each class grade, led by a team of three teachers. The program is characterized by four areas of change compared to the
traditional classroom: time - instead of switching between short lessons of about an hour each, the learning units take longer and are
of more meaningful time segments; place - the learning environment has evolved from a traditional classroom in which the tables are
organized in a particular structure facing the teacher, to a learning environment in a flexible, common and dynamic space. The
physical space of the class includes sub-spaces adapted to diverse teaching and learning methods, a meeting space, a space for
working in small groups, a space for personal work, formal and informal seating spaces. Classroom desks are on wheels so they can be
moved around in order to modulate and create learning and sitting environments as needed. The class is equipped with smart boards,
projectors and laptops, and the learning spaces are separated by partitions when working in small groups is required; learning topics the focus is on a broad subject, which involves asking, observing, studying, researching and creating instead of passing through
distinct topics that are taught by means of memorizing information; and the last area is learning and teaching methods – learning and
teaching are through collaborations between students and teachers rather than personal. During each lesson there is a team of three
teachers who guide the learning and in each field of learning the students work on projects throughout the year. The authentic project
engages them in solving a real-world problem or answering a complex question. They demonstrate their knowledge and skills by
developing a product or presentation for peers or parents. The emphasis in the class is on developing learning skills and transferring
the responsibility of learning to the students. The main focus is on how to learn and not what to learn. An example of a project in the
social sciences is presented in Appendix A.
The research questions were:
1 How does constructivist-based instruction (the innovative program) affect students' critical thinking and question-posing skills?
2 What differences, if any, in critical thinking and question-posing skill exist between the experimental and control groups?
A comparison was made between participating students in the innovative program and the traditional classes (who were taught
the same required subject content) at three points in time during the two year program. Pre-and post-case-based questionnaires were
designed to assess critical thinking and question-posing skill.
3.1. Participants
Participants in the two-year study were students who began in 9th grade (year one) and continued in 10th grade (year two). A
total of 192 questionnaires were analyzed, 114 filled out by students in the innovative program (IP) and 78 filled out by students in
the traditional class (TC; control group). Questionnaires were handed out at the beginning of 9th grade (pre-questionnaires) to 42
(37%) IP students and 29 (37%) TC students. The second evaluation point was done at the end of 9th grade when 34 (30%) IP
students and 22 (28%) TC students filled out the post-1 questionnaires. The third and final evaluation was done at the end of 10th
grade, when 38 (33%) IP students and 27 (35%) TC students filled out the post-2 questionnaires. Table 1 presents details about the
participants in each stage of the research.
3.2. Research tools
Unique research tools were developed to answer the research questions, adapted to assess the effectiveness of the pedagogic
change in teaching and learning methods of the innovative program. Pre-and post-case-based questionnaires were designed to assess
development of the skills of critical thinking and question-posing. Students were asked to formulate questions after reading a text and
to express a reasoned stance. At each point in time, two versions of the questionnaire were used in order to avoid repetition of content
and the effect of learning from the questionnaire. Thus, both IP and TC groups were subdivided into Groups A and B, with each group
receiving a different version of the questionnaire for each point in time. This method strengthened the validity of the questionnaires.
No statistically significant differences in the mean of the respondents were found in the t-test for independent variables between the
two questionnaire versions at each point in time. Thus, the difference between the versions does not constitute a source of variance in
the students' responses.
Each questionnaire had two sections, a short paragraph that examined the skill of question-posing and a section examining critical
thinking:
Table 1
Research population.
Traditional Class
Innovative Program
Time of measurement
29 (37%)
22 (28%)
27 (35%)
78 (100%)
42 (37%)
34 (30%)
38 (33%)
114 (100%)
Pre (Beginning of Grade 9)
Post-1 (End of Grade 9)
Post-2 (End of Grade 10)
Total
206
Thinking Skills and Creativity 29 (2018) 203–212
I. Sasson et al.
A Question-Posing: After reading the paragraph, students were asked to write all their questions arising from the text. Analysis of
this section included three indices (based on Dori and Herscovitz, 1999 and Zoller, 2002):
1 The number of questions phrased by students – the higher the number of questions raised, the more proficient the skill level
(20% of the total score).
2 Text dependency – the potential answers to the questions phrased by the students were examined and scores were given as
follows: 0 = no question was phrased by the student; 1 = the answer to the question phrased by the student appeared in the
text; and 2 = the answer to the question phrased by the student did not appear in the text (35% of the total score).
3 Quality of the question – the cognitive level of the questions was based on Bloom's taxonomy and scores were given as follows: 1
= Bloom's lowest two levels of Knowledge and Comprehension; 2 = Bloom's levels of Application and Analysis; and 3 =
Bloom's upper two levels of Synthesis and Evaluation (45% of the total score).
B Critical Thinking: The second section of the questionnaire dealt with the skill of argumentation, an essential element in the
development of critical thinking and decision-making skills. In the pre-questionnaire (first measurement period), students read a
statement of an expert. In the post-1 and post-2 questionnaires (second and third measurements), students read a suggested
solution to a dilemma. The students were asked to choose whether they agreed with the solution or the statement. A three-point
scale was used to assess students' critical thinking in the first measurement (based on Dori et al., 2003): 1 = supporting expert
opinion without questioning; 2 = the ability to question information that does not include enough data, but the argument is not
explained; and 3 = the ability to take a reasoned stance. A two-point scale was used to assess students' critical thinking in the
second and the third measurements: 1 = they agree or disagree with the suggested solution; and 2 = they agree or disagree with
the suggested solution supported by relevant reasoning.
Two different scales were used given the assumption that after completing the first questionnaire, students would easily understand that they should question expert opinion. Since these are the same students who completed the questionnaire three times
over two years, it was necessary to change the specific questionnaire section to ensure that the measuring tool did indeed examine the
skill development. Due to the use of two different scales, the comparative statistical analysis was performed on standardized scores.
The measurement method was validated by three researchers. Thirty questionnaires were analyzed by each researcher personally and
a matching rate of 86% was found. An example of one of the questionnaires can be found in the appendix to the article.
4. Results
4.1. Question-posing skill
Significant differences in the skill of question-posing were found at the start of the study in the pre-questionnaires between
students in the IP and TC groups. The IP student means were significantly higher for the indices of text dependency and cognitive
level according to Bloom's taxonomy. No significant differences were found between the groups for the index of the number of
questions that the students phrased after reading the text. Table 2 presents the results of the skill of question-posing in the prequestionnaire.
Since significant differences were found between the groups prior to the start of the innovative program, the assessment of the
skill of question-posing was done for each group in terms of its development over the two year study period. Fig. 1 presents the
results.
Anova test analysis for IP student scores indicate significant differences between measurements (F = 6.707, p = 0.02). A t-test
analysis of the pre-questionnaire and post-2 questionnaire results also found significant differences (t = 3.548, p = 0.001, effect size
Cohen's d = 0.794). Non-significant differences were found for the traditional class.
In the next stage of the analysis, changes over the two years were examined for each of the indices measuring the skill of questionposing. Fig. 2 presents the results of the cognitive level by Bloom's taxonomy.
Anova test analysis for IP student scores indicate significant differences (F = 10.911, p = 0.000). A t-test analysis of the prequestionnaire and post-2 questionnaire results also found significant differences (t = 4.415, p = 0.000, effect size Cohen's
d = 0.988). Non-significant differences were found for the traditional class. There were no significant changes for either the IP class
or the traditional class with respect to the other two question-posing skill indices: number of questions and text dependency.
Table 2
Means of question-posing skill – Pre questionnaire.
Mean Innovative Program
(S.D.)
Mean Traditional Class
(S.D.)
t-test Comparison
Effect Size
Cohen's d
No. of questions
Text dependency (Min = 0, Max = 2)
1.83)1.14)
1.50 (0.70)
1.38 (1.44)
(0.80) 0.98
–
0.698
Cognitive level by Bloom's taxonomy (Min = 1, Max = 3)
1.20 (0.64)
0.81 (0.67)
Question-posing skill
1.43)0.68)
0.98 (0.79)
n.s.
t=2.890
p = 0.005
t=2.428
p = 0.018
t=2.544
p = 0.013
207
0.586
0.614
Thinking Skills and Creativity 29 (2018) 203–212
I. Sasson et al.
Fig. 1. Means of question-posing skill – Sorted by class type.
Fig. 2. Means of cognitive level by Bloom's taxonomy – Sorted by class type.
4.2. Critical thinking
In contrast to question-posing, with regard to critical thinking, no significant differences were found between the groups in the
first measurement (prior to the start of the innovative program). Table 3 presents the results for critical thinking at the three
measuring periods with a comparative approach between the two groups.
Although no significant differences were found between classes in the pre-questionnaire, students in the innovative learning
environment demonstrated a significant advantage in critical thinking after two years. The effect size measured by Cohen's d is
relatively high. In the next stage of the data analysis, the development during the two years in critical thinking was examined for each
group. Table 4 presents the results.
A statistically significant improvement was found for IP students, whereas there were no significant changes among the traditional class.
5. Discussion
Active learning, particularly project-based learning, is perceived as potentially more interesting for students and teachers and
more successful than traditional approaches for developing high-order thinking skills. The goal of this research was to test this
perception by evaluating the effectiveness of an innovative, constructivist program for 9th and 10th graders on two skills: critical
thinking and question-posing. Pre- and post-case-based questionnaires were designed to assess these skills and a comparison was
made between students in the innovative program and students in the traditional class at three points in time over the two year study.
A total of 71 students participated in the research.
The research results indicate that students in the innovative learning environment demonstrated a significant advantage over
their peers in the traditional class in critical thinking after two years. For the skill of question-posing, the results revealed significant
differences between students in the innovative learning environment and the traditional class in the pre-questionnaires; therefore, we
Table 3
Means of critical thinking skill – Sorted by class type.
Pre (beginning of 9th grade)
Post-1 (end of 9th grade)
Post-2 (end of 10th grade)
Mean Innovative Program
(S.D.)
Mean Traditional Class (S.D.)
t-test
Effect Size
Cohen's d
−0.13 (0.99)
0.25 (1.04)
0.46 (0.60)
−0.28 (1.10)
−0.12 (1.05)
−0.42 (1.12)
n.s.
n.s.
t = 4.118 p = 0.000
–
–
1.037
208
Thinking Skills and Creativity 29 (2018) 203–212
I. Sasson et al.
Table 4
Means of critical thinking skill – Sorted by time of measurement.
Mean Innovative Program
(S.D.)
Mean Traditional Class (S.D.)
Pre (Beginning of 9th grade)
Post 1 (End of 9th grade)
Post 2 (End of 10th grade)
Anova test
−0.13 (0.99)
0.25 (1.04)
0.46 (0.60)
F = 4.481, p = 0.013
−0.28 (1.10)
−0.12 (1.05)
−0.42 (1.12)
n.s.
investigated the development over two years for each group separately. Students in the innovative class significantly improved their
skill, while non-significant differences were found for the traditional class between the pre and post-questionnaires. The findings of
the study indicate the ability to develop thinking skills among students in a relatively short time, even among high school students
previously educated by traditional learning methods. These results confirm the effectiveness of the constructivist approach for developing students' ability to ask questions, to query information that does not include sufficient data and to take a reasoned stance.
They reinforce the findings of previous studies (e.g. Hug, 2010; Lea et al., 2003; Loyens & Gijbels, 2008; Matthews, 2002) regarding
the contribution of constructivist learning environments to the development of high-order thinking skills, particularly critical
thinking and question-posing.
The main question in this research is about measuring the effectiveness or impact of new learning environment on the development of HOT thinking. The evidence was based on a quantitative tool that was applied in an experimental trial. The research was
produced directly by the questions and interests of professionals to provide an answer to an educational program in the field but the
contributions of the study are both for educators and researchers. Research evidence plays a relatively small part in directing professional decision making (Nelson & Campbell, 2017; Sharples, 2013). Usually, teachers' intuitive thoughts about teaching and
learning influence much more pedagogical practice. One of the contributions is to provide educators and other professionals with
arguments that can help them to justified (or not) their pedagogical practices and even more so to justified (or not) new intervention
programs.
Scientific work on the assessment of thinking skills can provide substantial evidence about the quality of teaching and learning,
therefore classroom data and research evidence can contribute to school improvement. "Evidence-based practice is about integrating
professional expertise with the best external evidence from research to improve the quality of practice" (Sharples, 2013, p. 7). Recent
years have been marked by the development of innovative pedagogical models in educational systems worldwide, with the goal of
replacing the traditional educational model. The new teacher, according to these models, designs active learning environments to
achieve a variety of goals at a high standard. There has been international confirmation of the importance of "evidence-based
education," or "evidence-informed practice" (EIP), understanding the "best-evidence," "best-practices," or "what works" (Borman,
Hewes, Overman, & Brown, 2003; Nelson & Campbell, 2017; Reynolds et al., 2014). This confirmation reinforces the potential
contribution of this study demonstrating the effectiveness of an innovative learning environment for the development of high-order
thinking skills. The study also demonstrates a methodology for assessing the skills of critical thinking and question-posing that can be
replicated in diverse learning environments by both researchers and educators.
Many educators do not have the knowledge and experience to develop and implement assessment tools. Test data is used mostly
for accountability, rather than to diagnose the needs of students and to improve education (Zwick et al., 2008). This is more complex
in the case of assessing high-order thinking skills. There is a continuing need for professional development in this area. The study
provides a tool for assessing the level of control of students in critical thinking and question-posing and can serve as a tool for
diagnosing or evaluating educational programs.
Several examples of teaching methods represented different applications of the constructivist approach have been found effective
in developing critical thinking and question-posing skills (Dori et al., 2003; Holmes et al., 2015; Kaberman & Dori, 2009). The
learning environment investigated in this study was characterized by collaborative project-based learning and co-teaching. The study
did not allow distinguishing between the unique effects of each pedagogical practice on the development of the thinking skills and
hence its limitations. This type of evaluation requires further research based upon observations and interviews.
Further research will also be required in order to understand the initial differences that were found in question-posing between
students in the innovative and traditional classes. The distribution of students into the innovative and traditional classes was based on
student choice and not on test classification or random assignment. Therefore, it is assumed that there were specific characteristics
that distinguished between the students. The source of the differences may be related to thinking dispositions. Tishman, Jay, and
Perkins, (1993) and Tishman (2000) distinguish between thinking skills and thinking dispositions, which are also known as habits of
mind. They define thinking dispositions as ongoing tendencies that guide intellectual behavior. Skills and dispositions are not always
in high correlation. Individuals can have high level ability in a specific skill but typically fail to use it. Cheng and Wan (2017)
investigated the effects of the classroom learning environment on critical thinking skills and disposition. The classroom learning
environment was characterized by seven dimensions: student negotiation, challenging task, multiple perspectives, shared control,
skeptical voice, uncertainty and personal relevance. Their results showed that the classroom learning environment had a stronger
relationship to critical thinking disposition than skills, and that critical thinking disposition mediates between the classroom learning
environment and critical thinking skills. Tishman et al. (1993) suggest the enculturation model for developing thinking dispositions,
with its three main components: role models for thinking, interaction in the classroom, and direct instruction for thinking skill.
Further research on the innovative program focused on classroom behaviors may shed light on this subject.
209
Thinking Skills and Creativity 29 (2018) 203–212
I. Sasson et al.
Appendix A
Project in Social Sciences:
In this project we will investigate how does age affect attitudes in relation to adolescence? We will build a survey with the
purpose of implementing the studied subject matter on the topic of adolescence. This survey is a short research exercise. The
project will include both team and personal work.
a Each group will formulate three questions for the survey. The questions should be related to the research objective. The
questions should be short, clear, explicit, objectively formulated, and relevant. The questions can be "open-ended" (when the
respondent freely expresses his or her position on the subject) or "closed-ended" (which includes multiple answer options from
which the respondents will choose the answer most appropriate to them). It is recommended to combine both methods in the
same questionnaire.
b Each group will propose three criteria according to which the quality of the questions in the questionnaire can be assessed. Each
group will receive the questions formulated by another group for a peer review according to the criteria proposed. Following the
evaluators' comments, the questions will be finalized in the questionnaire.
c After the completion of the joint class questionnaire, each student will collect the survey data from 4 people: yourself, a friend of
your age (not from the class) and 2 adults over the age of 40 (parents, uncles, etc.). Answers must be entered into a shared file.
d Each student will present the results collected in the form of a table and answer the following questions:
1 How do the findings correspond with the theoretical knowledge you have learned?
2 Compare the responses of the two adolescents in the survey (yours and your friend) to the responses of the two adults in the
survey. Differences and similarities within the themes must be pointed out.
e Each group will analyze three questions from the shared class data file while comparing adolescent and adult's attitudes. Draw a
conclusion arising from the analysis of the questions regarding the inquiry question.
f Choose how to present your findings and conclusions to your colleagues.
Appendix B
The following questionnaire has a text to read followed by two questions. The questionnaire is anonymous and has no impact on
your grade, but your answers may teach us a lot. We appreciate your serious response and cooperation.
The Discovery of Natural Gas in Israel
In recent years, the Israeli economy has undergone significant changes in the energy sector. Within a few years, natural gas
became the main and preferred source of energy for electricity generation and industrial use. Natural gas is a fossil fuel. It is
composed of a mixture of gases in which the main component is methane (CH4), which exists naturally between rock layers in
the depths of the earth. The advantages of natural gas as a source of energy are many: This is a cheap and clean energy source
that can be used in power plants, desalination plants, and other industries. The use of natural gas in the industry leads to
considerable savings in production costs. Natural gas may also be used as raw material for the chemical and petrochemical1
industries, as well as fuel for vehicles. As a result, the discoveries of natural gas alongside the coast of Israel in recent years
have great value, such as granting energy independence to the State of Israel, extensive economic development, and protecting
the environment and public health while reducing air pollution and greenhouse gas emissions.
The gas layout refers to the agreement between the State of Israel and the gas companies, which is intended to regulate the
management of the natural gas fields discovered in Israel by these companies, and in particular to ensure that the gas continues
to flow at a cost acceptable both to consumers and entrepreneurs; to ensure the development of gas production and
transmission systems; and to determine gas export quotas. Gas export quotas are supposed to ensure, on the one hand, that the
state's economy will have gas reserves for many years, while on the other hand, exporting gas will ensure revenues for the state
that will finance the continued development of the gas fields and the infrastructure of the gas transmission to the coastline.
Approval of the gas layout has aroused considerable public and political controversy. Its proponents argued that it would
promote the rapid development of gas reserves, raise taxes for the state, contribute to energy security, state security and
foreign relations. Opponents argued that the outline is beneficial mainly to the gas companies rather than to the citizens of the
state and in addition it may harm the competition in the gas industry and other industries, causing heavy financial damage to
the economy. There is also a concern that a long-term agreement with the gas companies will restrict future governments from
adhering to them.
1
Petrochemical industry: An industrial sector engaged in the production and production of chemical products originating mainly in fuels
such as petroleum, natural gas or coal.
1. Please write down all the questions that arise in your thoughts as a result of reading the text:
2. Here is a quote from the Minister of Energy. Please read it carefully, and answer the question that follows: "Gas supply will
help lower energy prices and lower the cost of living. The agreement could inject hundreds of billions of shekels over the next
20 years for the citizens of the State of Israel, for welfare, education and health budgets - a very important thing for the state".
Circle the sentence you relate to the most, and explain if necessary:
210
Thinking Skills and Creativity 29 (2018) 203–212
I. Sasson et al.
a
b
c
d
I
I
I
I
agree with his opinion since he is an expert in the field.
agree with his opinion, because______________________________
do not agree with his opinion because_____________________________
do not know whether I should agree with his opinion, because ___________
References
Atkins, M. J. (1993). Theories of learning and multimedia applications: An overview. Research Papers in Education, 8(2), 251–271.
Barton, K., & McCully, A. (2007). Teaching controversial issues… where controversial issues really matter. Teaching history, 127, 13.
Becker, H. J. (2000). Findings from the teaching, learning, and computing survey. Education Policy Analysis Archives, 8, 51.
Bednar, A. K., Cunningham, D., Duffy, T. M., & Perry, J. D. (1992). Theory into practice: How do we link. Constructivism and the Technology of Instruction: A
Conversation, 17–34.
Borman, G. D., Hewes, G. M., Overman, L. T., & Brown, S. (2003). Comprehensive school reform and achievement: A meta-analysis. Review of Educational Research,
73(2), 125–230.
Brierton, S., Wilson, E., Kistler, M., Flowers, J., & Jones, D. (2016). A comparison of higher order thinking skills demonstrated in synchronous and asynchronous online
college discussion posts. NACTA Journal, 60(1), 14.
Bruning, R. H., Schraw, G. J., & Ronning, R. R. (1999). Cognitive psychology and instruction. One Lake Street, Upper Saddle River, NJ 07458: Prentice-Hall, Inc.
Cheng, M. H. M., & Wan, Z. H. (2017). Exploring the effects of classroom learning environment on critical thinking skills and disposition: A study of Hong Kong 12th
graders in Liberal Studies. Thinking Skills and Creativity, 24, 152–163.
Costa, A. (2008). The thought-filled curriculum. Educational Leadership, 65(5), 20–24.
Craft, A. R., & Chappell, K. A. (2016). Possibility thinking and social change in primary schools. Education 3-13, 44(4), 407–425.
David, J. (2008). Project based learning. Educational Leadership, 65(5), 80–82.
Dori, Y. J., & Herscovitz, O. (1999). Question-posing capability as an alternative evaluation method: Analysis of an environmental case study. Journal of Research in
Science Teaching, 36(4), 411–430.
Dori, Y. J., & Sasson, I. (2013). A three-attribute transfer skills framework–part I: Establishing the model and its relation to chemical education. Chemistry Education
Research and Practice, 14(4), 363–375.
Dori, Y. J., & Sasson, I. (2008). Chemical understanding and graphing skills in an honors case‐based computerized chemistry laboratory environment: The value of
bidirectional visual and textual representations. Journal of Research in Science Teaching, 45(2), 219–250.
Dori, Y. J., Tal, R. T., & Tsaushu, M. (2003). Teaching biotechnology through case studies - can we improve higher order thinking skills of nonscience majors? Science
Education, 87(6), 767–793.
Dwyer, C. P., Hogan, M. J., & Stewart, I. (2011). The promotion of critical thinking skills through argument mapping.
Ennis, R. H. (1989). Critical thinking and subject specificity: Clarification and needed research. Educational Researcher, 18(3), 4–10.
Ennis, R. H. (1985). A logical basis for measuring critical thinking skills. Educational Leadership, 43(2), 44–48.
Epstein, A. S. (2008). An early start on thinking. Educational Leadership, 65(5), 38–42.
Ertmer, P. A. (2005). Teacher pedagogical beliefs: The final frontier in our quest for technology integration? Educational Technology Research and Development, 53(4),
25–39.
Ertmer, P. A., & Newby, T. J. (1993). Behaviorism, Cognitivism, Constructivism: Comparing critical features from an instructional design perspective. Performance
Improvement Quarterly, 6(4), 50–72.
Evans, D. J., Zeun, P., & Stanier, R. A. (2014). Motivating student learning using a formative assessment journey. Journal of Anatomy, 224(3), 296–303.
Facione, P. A. (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction—The Delphi reportMillbrae, CA: California
Academic Press.
Frid, S. (2000). Constructivism and reflective practice in practice: Challenges and dilemmas of a mathematics teacher educator. Mathematics Teacher Education and
Development, 2, 17–33.
Gambrill, E. (2006). Critical thinking in clinical practice: Improving the quality of judgments and decisions. John Wiley & Sons.
Holmes, J., & Clizbe, E. (1997). Facing the 21st Century. Business Education Forum, 52(1), 33–35.
Holmes, N. G., Wieman, C. E., & Bonn, D. A. (2015). Teaching critical thinking. Proceedings of the National Academy of Sciences of the United States of America, 112(36),
11199–11204. https://doi.org/10.1073/pnas.1505329112.
Hug, T. (2010). Radical constructivism mainstreaming: A desirable endeavor? Critical considerations using examples from educational studies and learning theory.
Constructivist Foundations, 6(1).
Kaberman, Z., & Dori, Y. J. (2009). Metacognition in chemical education: Question posing in the case-based computerized learning environment. Instructional Science,
37(5), 403–436.
Karmon, A. (2007). Institutional organization of knowledge: The missing link in educational discourse. Teachers College Record, 109, 603–634.
King, F. J., Goodson, L., & Rohani, F. (1998). Higher order thinking skills. Retrieved from:Center for Advancement of Learning and Assessmenthttp://www.cala.fsu.edu/
files/higher_order_thinking_skills.pdf.
Lea, S. J., Stephenson, D., & Troy, J. (2003). Higher education students’ attitudes to student-centred learning: beyond’educational bulimia’? Studies in Higher Education,
28(3), 321–334.
Lee, H. (1980). The effects of review questions and review passages on transfer skills. The Journal of Educational Research, 73(6), 330–335.
Lewis, A., & Smith, D. (1993). Defining higher order thinking. Theory Into Practice, 32(3), 131–137.
Liu, S. H. (2011). Factors related to pedagogical beliefs of teachers and technology integration. Computers & Education, 56, 1012–1022.
Loyens, S. M., & Gijbels, D. (2008). Understanding the effects of constructivist learning environments: Introducing a multi-directional approach. Instructional Science,
36(5-6), 351–357.
Madhuri, G. V., Kantamreddi, V. S. S. N., & Prakash Goteti, L. N. S. (2012). Promoting higher order thinking skills using inquiry-based learning. European Journal of
Engineering Education, 37(2), 117–123.
Mansilla, V. B., & Gardner, H. (2008). Disciplining the mind. Educational Leadership, 65(5), 14–19.
Matthews, M. R. (2002). Constructivism and science education: A further appraisal. Journal of Science Education and Technology, 11(2), 121–134.
Nagowah, L., & Nagowah, S. (2009). A reflection on the dominant learning theories: Behaviourism, cognitivism and constructivism. The International Journal of
Learning, 16(2), 279–286.
Nelson, J., & Campbell, C. (2017). Evidence-informed practice in education: Meanings and applications. Educational Research, 59(2), 127–135.
Perkins, D. N. (1986). Thinking frames. Educational leadership, 43(8), 4–10.
Perkins, D., & Ritchhart, R. (2004). When is good thinking. Motivation, emotion, and cognition. Integrative perspectives on intellectual functioning and development,
351–384.
Resnick, L. (1987). Education and learning to think. Washington, D.C: National Academy Press.
Resnick, L. B., Michaels, S., & O’Connor, C. (2010). How (well-structured) talk builds the mind. In R. Sternberg, & D. Preiss (Eds.). From genres to context: New discoveries
about learning from education research and their applications (pp. 163–194). New York: Springer.
Reynolds, D., Sammons, P., De Fraine, B., Damme, J. V., Townsend, T., Teddlie, C., ... Stringfield, S. (2014). Educational effectiveness research (EER): A state-of-the-art
211
Thinking Skills and Creativity 29 (2018) 203–212
I. Sasson et al.
review. School Effectiveness and School Improvement, 25, 197–230.
Rivet, A. E., & Krajcik, J. S. (2004). Achieving standards in urban system reform: An example of a sixth grade project-based science curriculum. Journal of Research in
Science Teaching, 41, 669–692.
Roschelle, J., Penuel, W. R., Yarnall, L., Shechtman, N., & Tatar, D. (2005). Handheld tools that ‘Informate’assessment of student learning in Science: a requirements
analysis. Journal of Computer Assisted Learning, 21(3), 190–203.
Rosenfeld, M., & Rosenfeld, S. (2006). Understanding teacher responses to constructivist learning environments: Challenges and resolutions. Science Education, 90,
385–399.
Sandholtz, J. H., & Reilly, B. (2004). Teachers, not technicians: Rethinking technical expectations for teachers. Teachers College Record, 106, 487–512.
Sasson, I., & Dori, Y. J. (2015). A three-attribute transfer skills framework–part II: applying and assessing the model in science education. Chemistry Education Research
and Practice, 16(1), 154–167.
Sharples, J. (2013). Evidence for the frontline. Accessed July 25, 2018London: Alliance for Useful Evidence [online]. http://www.alliance4usefulevidence.org/assets/
EVIDENCE-FOR-THE-FRONTLINE-FINAL-5-June-2013.pdf.
Sharples, M., Adams, A., Alozie, N., Ferguson, R., FitzGerald, E., Gaved, M., & Roschelle, J. (2015). Innovating pedagogy 2015: Open university innovation report 4.
Swartz, R. (2008). Energizing learning. Educational Leadership, 65(5), 26–31.
Thomas, J. W. (2000). A review of research on project-based learning.
Tigelaar, D. E., & Beijaard, D. (2013). Special issue: formative assessment and teacher professional learning. Teachers and Teaching Theory and Practice, 19(2), 109–114.
Tishman, S. (2000). Why teach habits of mind? In A. Costa, & B. Kallick (Eds.). Discovering and exploring habits of mind (pp. 41–52). ASCD.
Tishman, S., Jay, E., & Perkins, D. N. (1993). Teaching thinking dispositions: From transmission to enculturation. Theory Into Practice, 32(3), 147–153.
Trevisan, M. S. (2007). Evaluability assessment from 1986 to 2006. The American Journal of Evaluation, 28(3), 290–303.
Tsaparlis, G., & Papaphotis, G. (2009). High‐school Students’ Conceptual Difficulties and Attempts at Conceptual Change: The case of basic quantum chemical
concepts. International Journal of Science Education, 31(7), 895–930.
Von Glaserfeld (1991). Knowing without metaphysics: Aspects of the radical constructivist position. In F. Steier (Ed.). Research and reflecting (pp. 12–29). London: Sage.
Windschitl, M. (2002). Framing constructivism in practice as the negotiation of dilemmas: An analysis of the conceptual, pedagogical, cultural, and political challenges
facing teachers. Review of Educational Research, 72(2), 131–175.
Zohar, A., & David, A. B. (2009). Paving a clear path in a thick forest: a conceptual analysis of a metacognitive component. Metacognition and Learning, 4(3), 177–195.
Zoller, U. (2002). Algorithmic, LOCS and HOCS (chemistry) exam questions: Performance and attitudes of college students. International Journal of Science Education,
24(2), 185–203.
Zoller, U., & Tsaparlis, G. (1997). Higher and lower-order cognitive skills: The case of chemistry. Research in Science Education, 27(1), 117–130.
Zwick, R., Sklar, J. C., Wakefield, G., Hamilton, C., Norman, A., & Folsom, D. (2008). Instructional tools in educational measurement and statistics (ITEMS) for school
personnel: Evaluation of three web-based training modules. Educational Measurement Issues and Practice, 27(2), 14–27.
212