An Inquiry on Higher-Order Thinking Skills of Students who had Study the
Unit “Treasures of the Sea” Within “Science and Technology for All”
Danay Rakefet, The Department of Education in Technology and Science, Technion - Israel Institute of Technology.
danay@techunix.technion.ac.il.
Lazarowitz Reuven, The Department of Education in Technology and Science. Technion - Israel Institute of
Technology. rlazar@tx.technion.ac.il.
Abstract One of the main goals in science education is to provide students who do not
major in science with scientific and technology literacy and mastery of problem solving
skill, which require formal reasoning ability.
In this study a learning unit "Treasures of the Sea" was developed and taught to 10 th to
11th grade students. The learning unit was written in a STS approach based on the biome
level of biological organization including modes of instruction.
In this study, non-major students were investigated on mastery of problem solving skills
related to the "Treasures of the Sea" module. Data were collected through quantitative
and qualitative methods. Written/ open-ended tasks tests were developed for assessing
the students’ performance on problem solving skills. The results revealed the progress
students made while studying the module and the difficulties that non-major students
encounter while learning integrated STS topics on the biome level of biological
organization of the sea environment. The findings of the study provide new approach to
master and assess open-ended problem solving skills, and suggest that science
instruction should put more emphasis on the acquire of high cognitive skills using the
strategy of solving problems step by step respectively to a specific biological topic,
rather than the holistic mode. Teaching complex scientific ideas and developing high
cognitive skills can provide students with the useful tools needed in order to function on
a highly scientific and technology society life.
Keywords: problem solving, cognitive skills, cognitive reasoning, sea environment
Introduction
A major goal in science education is to develop the cognitive skills of all the students. Researchers are dealing with
the question of how to include the mastery and understanding of science and technology literacy, and how to foster
thinking skills as a part of the science education of every citizen. (Zohar & Dori, 2003)
The Israeli curriculum “Science and Technology for All” intends to improve scientific and technological literacy of
high school students who choose do not major in one of the science or technological disciplines. These students
display a heterogeneous population. Studies have shown that 60% of the students in the high school are in the
cognitive concrete operational stage and only 40% are in their formal stage, while skills such as: solving problems,
critical thinking and creative thinking require formal reasoning operations (Shemesh & Lazarowitz, 1988; Valanides,
1996).
Solutions to the problem can be offered by developing appropriate curriculum materials and teaching strategies.
Therefore identification of curriculum materials that support this learning and help teachers to build their own
content and pedagogical knowledge is essential. (Stern & Roseman, 2004).
The module “Treasures of the Sea “ (Danay, Stoler, &. Tal, 2004).
The module described the environment of the sea from different aspects – oceanography, physics ,chemistry,
biology, aqua-culture and by human impact on the sea environment.
The originality of this module is in the integration of the sea environment topics in a biological level of organization,
the biome, (as described by Schwab, 1963), blending with pedagogical modes of instruction based on the Science
Technology and Society (STS) approach.
The study goals
In this study, the academic achievement, the mastering of problem solving skills and learning difficulties of non
major science students in the context of the high ecological level, the biome, were investigated and evaluated, while
the module was taught.
1
Methodology
The study was conducted in seven classes, with 150 students (16 to 17 years old) .The students learned the module
during 50 periods.
Data were collected with instruments using qualitative and quantitative methods. The instruments enabled the
identification of the following:
Students’ background: school sector, class level, gender and stages of cognitive development. Students’ performance
on VTGT test (Shemesh & Lazarowitz, 1988) was used for assessing students' formal reasoning stages.
The dependent variables were investigated with 1. A test for assessing the academic achievement. 2. A written/ openended tasks test for assessing the students’ performance on problem solving skills. 3. Interviews carried out with the
students. 4. Classroom observations and conversations with teachers
Curriculum material: The module was analyzed for the presence of tasks on solving problems skills and their
relevancy to the subject of the sea environment, the biome, presented in a STS approach. (Bybee, 1987).
Teaching strategies: teachers' personal and professional background, teachers’ in- service training, the modes of their
instruction, and their behaviors in the classroom lessons’ by the researcher's observations.
In this summary we focus specifically on the following theme: solving a problem step by step or in a holistic mode.
In the written/ open-ended tasks students were asked to solve the problem step by step or in a holistic mode. A case
study was presented with six step by step assignments, which included: posing questions, identify outcomes of the
problems, identify problems, think of new problems that may occurred, formulate possible solutions and implications
of the solutions. In a holistic mode, students were asked to write a letter to a friend about the problems and to pay
attention to the entire problem solving procedure.
The pre- and posttests were similar in their structure and number of items, but not identical. Three science educators
validated the tests and decided on the scoring scheme.
Scoring rubrics is the method in this research for evaluating student work. It guides the analysis of the processes of
students' written/ open-ended tasks. Scoring rubric has several components: dimensions that serve as the basis for
judging the student response. Every dimension includes criteria for assessing. Criteria have definitions and examples
to clarify the meaning of each criterion (Birenbaum, 1997).
Results
Students’ performance on VTGT test (Shemesh &Lazarowitz, 1988) was used for assessing students' formal
reasoning stages and the result are presented in Figure 1.
Formal
13%
Concrete
48%
Transitional
39%
Figure 1. Percentages of students on the cognitive reasoning stages from the sample study.
2
The results show:
Percentages of students, at the different stages of cognitive development: 48% concrete, 39% transitional and 13%
formal (N=112).
Data were analyzed for mastery of skills, which are included in the dimensions of problem solving procedure, system
thinking, and cognitive level according to the Bloom taxonomy (Bloom, 1956). None significant differences between
students in the cognitive reasoning stages.
Students’ mean scores in the achievement of this dimensions, step by step or holistic mode pre and post-test were
treated with a Mann-Whitney tests, and are presented in tables 1-3.
Table 1
Mean Scores and Mann-Whitney Tests for Problem Solving Skills Step by Step or Holistic Mode in Pre and
Post-Test in The Dimension: Problem Solving Procedure.
Criteria
Pre-test.
Post-test
Dimension
Problem solving procedure
Step by
Step
M (sd)
Holistic
Mode
M (sd)
Z(P)
Step by
Step
M (sd)
Holistic
Mode
M (sd)
Z(P)
Asking question
4.04
(1.44 )
0.15
(0.49 )
-13.720
(0.000)
.4.4
(1.38 )
1400
(0.36 )
-13.341
(0.000)
Identify outcomes of the
problems
0401
(0.56)
041.
(1.43)
-2.171
(0.030)
Identify problems
14.1
(0.46)
14.4
(0.50)
-4.249
(0.000)
1414
(0.38)
14..
(0.50)
-4.678
(0.000)
Identify the main problem
14.0
(0..1)
14.0
(0.4.)
-3.288
(0.001)
14.0
(0..0)
14.1
(0.40)
-2.860
(0.004)
Raise new problems
04.1
(0.0.)
0404
(0.0.)
-9.031
(0.000)
1440
(0.00)
14.0
(0.11)
-8.446
(0.000)
Formulate possible solutions
04.0
(0...)
14..
(1.00)
-5.471
(0.000)
041.
(0.0.)
1401
(1.01)
-6.150
(0.000)
Implications of the solutions
0400
(0..0)
14..
(1..0)
-7.629
(0.000)
.400
(0.04)
14.1
(1..1)
-8.586
(0.000)
1401
144.
(1..1)
-4.697
140.
1441
-4.997
(0.000)
(1.44)
(1..0)
(0.000)
No answer
(1.40)
n.s
The results show:
In the dimension of problem solving procedure (table 1) in the criteria: asking question, identify problems, identify
the main problem, formulate possible solutions and implications of the solutions, when students used the step by step
approach they achieved significantly higher mean scores than when they used the holistic mode. In the criteria:
identify outcomes of the problems and raise new problems students achieved significantly lower mean scores.
3
Table 2
Mean Scores and Mann-Whitney Tests for Problem Solving Skills Step by Step or Holistic Mode in Pre and
Post-Test in The Dimension: Cognitive Levels.
Dimension
Criteria
Pre-test.
Step by
Step
M (SD)
041.
(0..0)
Holistic
Mode
M (SD)
14.4
(1...)
Low Cognitive level
140.
(0...)
No answer
1401
High cognitive level
Cognitive level
(1.40)
Post-test
Z(P)
-3.447
(0.001)
Step by
Step
M (SD)
04..
(0...)
Holistic
Mode
M (SD)
140.
(1..1)
Z(P)
-5.423
(0.000)
14..
(1..1)
-3.499
(0.000)
14..
(0.00)
14.1
(1...)
-4.077
(0.000)
144.
(1..1)
-4.697
140.
1441
-4.997
(0.000)
(1.44)
(1..0)
(0.000)
The results show:
In the dimension of cognitive level (see table 2) when students used the step by step approach, they achieved
significantly higher mean scores than they had used the holistic mode. In the criteria: no answer, students achieved
significantly higher mean scores when they used the holistic mode than when they used the step by step approach.
Table 3
Mean Scores and Mann-Whitney Tests for Problem Solving Skills Step by Step or Holistic Mode in Pre and
Post-Test in The Dimension: System Thinking.
Dimension
Criteria
Pre-test.
Step by
Step
M (SD)
Holistic
Mode
M (SD)
Post-test
Z(P)
Step by
Step
M (SD)
Holistic
Mode
M (SD)
Z(P)
System thinking
Society
n.s
n.s
Science
n.s
n.s
Technology
n.s
n.s
Aspect ccombination
144.
(1..1)
1448
(1.50)
-2.894
(0.004)
1449
(1.50)
1461
(1.49)
-2.277
(0.050)
The results show:
In the dimensions of system thinking (see table 3) there were no significant differences in the criteria: society,
science and technology. In the criteria: aspect combination (two of: society, science and technology) students
achieved significantly lower mean scores when they used the step by step approach than when they used the holistic
mode.
4
Discussion and implications
The research has brought up a variety of issues related to teaching higher order cognitive skills for non-majors in
science (Zohar, & Dori, 2003). These students had difficulties in the holistic problem solving approach. But students
made progress after learning the module.
In most of the criteria, there were higher mean scores in using the strategy of solving problems step by step. Higher
mean scores were in the holistic problem solving approach, only when students had to use low cognitive skills or
complexity thinking: identification of outcomes of a problem requires to be a good knowledge teller. Raising new
problems and aspect combination of system thinking, required generating new ideas, analysis, synthesis, and
evaluation and integration of new information related to prior knowledge.
Scientific literacy is used in problem solving. Promote scientific literacy could be seen as part of the larger project of
literacy development and promotion. If the reader has failed to grasp the scientific meaning beyond the surface
content level he/she failed to grasp science. (Norris & Phillips, 2003)
Most of the problems in our lives are open-ended problems. According to Hurst & Milkent, (1996), unsuccessful
students tend to tackle a problem haphazardly and rarely review their answers for inconsistencies. It can be
hypothesized that, if students are given the experience of working on open-ended problems step by step in learning
science, they would be better prepared and equipped to face real-world challenges in their future.
Educational significance of the research
The study provides new approach on assessing open- ended problem solving skills.
These findings suggest that science instruction should put more emphasis: on developing problem solving skills
using the strategy of solving problems step by step respectively to the specific problem; on teaching complex
scientific ideas, since students who receive this mode of instruction may be provided with the opportunity to think
independently and in creative and original ways.
References
Birenbaum, M. (1997). Alternatives in Assessment. Israel: Ramot Publishers
Bybee, R. W. (1987). Science education and Science-Technology-Society (STS) Theme. Science Education, 71(5),
667-683.
Bloom, B.S. (1956). Taxonomy of educational objectives: the classification of educational goals. London:
Longmans.
Danay, R., Stoler, A. & Tal, T. (2004). Treasures of the Sea. Department of Education in Technology and Science,
Technion IIT, Haifa, Israel (in Hebrew).
Hurst, R. W. & Milkent, M. M. (1996). Facilitating successful prediction problem solving in biology through
application of skill theory. Journal of Research in Science Teaching, 33(5), 541-552.
Norris. S. P. &, Phillips. L. M. (2003). How literacy in its fundamental senses is central to scientific literacy. Science
Education, 87, 224– 240.
Schwab, J.J. (1963). Biology Teachers' Handbook. New York; John Wiley and Sons, Inc.
Shemesh, M. & Lazarowitz, R. (1988). The interactional effects of students' cognitive levels and test characteristics
on the performance of formal reasoning tasks. Research in Science & Technological Education, 6(1), 79-89
Stern, L. & Roseman, E. (2004). Can middle-school science textbooks help students learn important ideas? findings
from project 2061’s curriculum evaluation study: life science. Journal of Research in Science Teaching. 41(6),
538–568.
Valanides, N. C. (1996). Formal reasoning and science teaching. School Science and Mathematics, 96(2), 99–107.
Zohar, A. & Dori, Y. (2003). Higher order thinking skills and low achieving students: Are they mutually exclusive?
Journal of the Learning Sciences. 12(2), 145-182.
Appendix
1: Example of one item from the academic achievement test

What is the source of the oxygen in the water that is required for the seaweed?
1.
2.
3.
4.
The seaweed detaches the water molecules, and gain oxygen.
The seaweed receives oxygen from the liquefied oxygen in the water.
The seaweed uses the excess oxygen within their body's, the outcome of the
photosynthesis process.
The seaweed does not require oxygen because they do not breathe underwater.
5
2: Description of A written/ open-ended tasks test for assessing the students’ performance on problem solving
skills.
A case study was presented with six step by step assignments, which included: posing questions, identify outcomes
of the problems, identify problems, think of new problems that may occur, formulate possible solutions and
implications of the solutions. In a holistic mode, students were asked to write a letter to a friend about the problems
and to pay attention to the entire problem solving procedure.
The pre- and posttests were similar in their structure and number of items, but not identical.
3: Example of one item from the video-taped group test (VTGT) (Shemesh &Lazarowitz, 1988) that used for
assessing students' formal reasoning stages.

There are two red squares and four yellow squares in a bag. What are the odds of pulling out a red
square?
1.
2.
3.
4.
5.
1 out of 1 (100%)
1out of 2 (50%)
1 out of 3 (33.3%)
1 out of 4 (25%)
1 out of 6 (16.66%)
Explain:
______________________________________________________________________________________________
________________________________________________________________________________ ______________
______________________________________________________________________________________________
6