Paper in “Science Education: Past, Present and Future” edited by Reinders
Duit et. al. (Kluwer Academic Publishers: Dortrecht, 2001) 331-336
Point and Set Paradigms in Students’ Handling of
Experimental Measurements
Saalih Allie and Andy Buffler
Department of Physics, University of Cape Town, South Africa
Fred Lubben and Bob Campbell
Department of Educational Studies, University of York, UK
Abstract
The procedural understanding of first year university students before and after instruction
has been investigated in the context of experimental work in physics. A written instrument
was used to probe the students’ ideas about data collection, processing and comparison.
The responses of the students are analysed in terms of “point” and “set” paradigms which
are proposed as a framework for evaluating the effectiveness of laboratory curricula.
Introduction
Experimentation and measurement are fundamental to knowledge production in
the natural sciences. Meaningful engagement by students in scientific activities that
are experimentally based requires an understanding of the reasons for the procedures
that are followed. However, a growing body of work indicates that students at both
school (Coelho and Séré, 1998) and university (Evangelinos et al., 1999) carry out
the procedures without such a basic understanding. In a study of undergraduate
physics students, for example, a clear disjuncture between virtuosity in applying the
formalism of data analysis and the level of understanding were observed (Séré et al.,
1993). The current work forms part of a research programme in which procedural
understanding of experimental work of entering undergraduate science students at
the University of Cape Town is being documented. The ultimate aim of the exercise
is to use the findings to inform the development of an introductory physics
laboratory curriculum. This paper reports on procedural understanding in the context
of experimental work in physics of a group of students both at entry to the university
and after a twelve-week laboratory course. The laboratory course in question is part
of the physics component of the Science Foundation Programme (SFP) which was
set up at the University of Cape Town to assist educationally disadvantaged
students. Details of the SFP and the laboratory course are contained in Allie and
Buffler (1998).
Since most of the SFP students have little or no first hand practical experience, a
prime aim of the laboratory course is to develop the notion of measurement. Aspects
of data collection and data processing are addressed by exercises such as drawing up
tables, taking several measurements of a quantity, plotting graphs, fitting straight
lines, and calculating the mean and the standard deviation from the statistical
formulae as well as graphically from a Gaussian curve. The idea of spread in data is
introduced by getting the class to measure the time of travel of a sound pulse over a
given dis tance. The readings are processed to form a distribution (a Gaussian curve
results) from which the key ideas of mean and uncertainty are introduced. The
laboratory course consists of weekly three-hour sessions over twelve weeks. About
half of this time is spent in the laboratory carrying out experiments while the
remainder is used for the exercises described.
Methodology
The research instrument comprised a set of nine written questions (probes) based
on those developed for a previous study (Allie et al., 1998). All these probes related
to the same posited experiment which was presented as follows, together with a
detailed diagram (not shown here). “An experiment is being performed in the
Physics laboratory. A wooden slope is clamped near the edge of a table. A ball is
released from a height h above the table as shown in the diagram. The ball leaves
the slope horizontally and lands on the floor a distance d from the edge of the table.
Special paper is placed on the floor on which the ball makes a small mark when it
lands. The students have been asked to investigate how the distance d on the floor
changes when the height h is varied. A metre stick is used to measure d and h”.
The situation was also demonstrated using a large-scale model. The probes focused
on decisions to be made while collecting data, processing data and comparing two
different sets of measurements of the same quantity. Each probe was of the same
form as the example shown in fig. 1 below. Thus, a situation was presented where a
procedural decision was required and a number of alternative actions (A, B, C) were
suggested. Most importantly, a detailed reason for each choice was requested. The
probes were answered under formal examination-type conditions, strictly in the
sequence presented. As each answer sheet was completed it was placed into an
envelope and never reviewed by the student. The instrument was completed by 70
students before and after they had completed the laboratory course.
Figure 1. Example of a probe: in this case the reasons for repeating measurements.
The students work in groups on the experiment. Their first task is to determine d when
h = 400 mm. One group releases the ball down the slope at a height h = 400 mm and, using
a metre stick, they measure d to be 436 mm.
The following discussion then takes place between the students.
Why? We’ve got the
result already. We do
not need to do any
more rolling.
I think that we should
roll the ball a few more
times from the same
height and measure
d each time.
I think we should
roll the ball down
the slope just one
more time from the
same height..
A
B
With whom do you most closely agree? (Circle ONE):
Explain your choice below.
C
A
B
C
Analysis
The analysis of the probes consisted of categorizing the student responses
according to the answer choice (A, B, C) together with the different types of
reasoning put forward by the students. The coding of the responses was undertaken
using an alphanumeric scheme which was developed and tested previously (Allie et
al., 1998). This enabled the underlying reasoning to be identified for each student.
Earlier work (Buffler et al., 1998) suggested that the actions and reasoning
employed by students could be classified into two groups by defining two
constructs, namely a “point” paradigm and a “set” paradigm as discussed below.
Using this framework, the pre- and post-test results were analysed by looking for
patterns across the three areas that the probes addressed, namely, data collection,
data processing and data set comparison.
The point paradigm is characterised by the notion that each measurement
results in a single, “point-like” value which could in principle be the true value. As
a consequence each measurement is independent of the others and the individual
measurements are not combined in any way. In its most extreme form, this way of
thinking manifests itself in the belief that only one single measurement is required to
establish the true value, as indicated in the work of Séré et al. (1993). Responses
were coded as being associated with the point paradigm when, for example: (a) it
was stated that measurements are repeated in order to find a recurring value or to
perfect the measuring skill in order to finally take one ‘perfect’ measurement; (b) a
specific measurement was selected (e.g. the highest, the recurring, the first or the
last) to represent a series of numerical readings; (c) specific points (such as the
origin, the extreme points or any three aligned points) were chosen through which to
draw a straight line to represent a collection of plotted points; or (d) two sets of data
were contrasted either by comparing individual measurements in the sets, or by
treating the mean values of the data sets as points to be compared.
The set paradigm is characterised by the notion that each measurement is only
an approximation to the true value and that the deviation from the true value is
random. As a consequence, a number of measurements are required to form a
distribution that clusters around some particular value. The best information
regarding the true value is obtained by combining the measurements using
theoretical constructs in order to describe the data collectively. The operational tools
that are available for this purpose include the formal mathematical procedures that
can be used to characterise the set as a whole, such as calculating the mean and the
standard deviation. In turn, these quantities become tools for making comparisons
with other data-sets or theory. Responses were coded as being associated with the
set paradigm when, for example: (a) it was stated that repeating measurements was
aimed at taking a mean; (b) the mean and the spread were calculated to represent the
data; (c) a ‘line of best fit’ that took account of all points was drawn for plotted data;
or (d) different sets of measurements were contrasted by comparing the degree of
overlap of the intervals defined by the mean and some measure characterising the
spread of the data.
Results
Table 1 summarises the results from the pre- and post-tests with regard to
students’ understanding about repeating measurements during data collection (3
probes). It shows that before instruction the large majority of students (76%)
subscribed to the point paradigm while after instruction there appeared to be a large
shift (16% to 71%) towards the set paradigm. However, it is not clear that these
students have embraced the set paradigm as a whole. For example, many students
indicated that the purpose of repeating measurements is to allow for a mean to be
generated (rather than a mean being a way of dealing with the inherent scatter in the
data). This suggests there is a strong possibility that elements of the set paradigm are
being used by rote or on an ad hoc basis. The degree to which this is the case
requires the combined analysis of the other probes.
Table 1.
Students’ use of paradigms for
data collection.
Paradigm
before
instruction
Point
paradigm
Set
paradigm
Not
codeable
Total
Paradigm after instruction
Point
Set
Not
paradigm
paradigm
codeable
9
40
4
(13%)
(57%)
(6%)
4
6
1
(6%)
(9%)
(1%)
2
4
0
(3%)
(5%)
(0%)
15
50
5
(21%)
(71%)
(7%)
Total
53
(76%)
11
(16%)
6
(8%)
70
(100%)
Table 2 summarises the pre- and post test findings for 3 probes dealing with the
comparison between two data-sets. The first of the three probes required students to
compare two sets of data with the same mean but different scatter, while the second
probe provided two sets of data with different means but the same (overlapping)
spread. The third probe presented two data-sets with different means and different
but overlapping spreads. In Table 2 students are grouped according to whether or not
their responses across the three probes were consistent with the set paradigm. As
expected from the background of the students, none were classified as using the set
paradigm consistently prior to instruction. After instruction only 26% responded
consistently in terms of the set paradigm while more than two thirds (70%) resorted
to both paradigms, possibly indicating either rote or ad hoc application of the
elements associated with the set paradigm.
Table 2.
Students’ use of paradigms
when comparing data-sets.
Paradigm
before
instruction
Mixed
paradigms
Consistent set
paradigm
Not
codeable
Total
Paradigm after instruction
Mixed
paradigms
48
68%)
0
(0%)
1
(1%)
49
(70%)
Consistent
set paradigm
18
(26%)
0
(0%)
0
(0%)
18
(26%)
Not
codeable
3
(4%)
0
(0%)
0
(0%)
3
(4%)
Total
69
(98%)
0
(0%)
1
(1%)
70
(100%)
Tables 3 and 4 compare various aspects of the post-probes. In Table 3 the post-
probe results of Table 2 (data-set comparison) are contrasted with the combined
results of the post-probes dealing with data collection (one probe) and data
processing (two probes). One of the data processing probes required a mean to be
calculated from a set of numerical data, while the other required a straight line to be
drawn to a set of graphical data. It is interesting to note that there appears to be a
strong link between the paradigms used for these two probes. For example, students
who joined individual data points (i.e. did not fit a straight line to the data as a
whole) often evidenced point paradigm use in the other probe by choosing the
recurring value to represent the data rather than calculating a mean. (Space
limitations preclude showing the evidence in detail). Table 3 shows that only about a
fifth (21%) of the students based their responses on the set paradigm for all of data
collection, data processing and data-set comparison and that the largest group (37%)
were inconsistent in their use of the paradigms.
Table 3.
Students’ use of paradigms
for data collection /
processing and data-set
comparison after instruction.
Paradigms
used in
data-set
comparison
Mixed
paradigms
Consistent
set paradigm
Not
codeable
Total
Paradigms used in data collection / processing
Consistent
point
paradigm
9
(13%)
0
(0%)
0
(0%)
9
(13%)
Mixed
paradigms
25
(37%)
3
(4%)
1
(1%)
29
(42%)
Consistent
set
paradigm
13
(18%)
15
(21%)
2
(3%)
30
(43%)
Not
codeable
Total
2
(3%)
0
(0%)
0
(0%)
2
(3%)
49
(70%)
18
(26%)
3
(4%)
70
(100%)
Table 4 shows results for students who have been classified on the basis of all
the probes discussed thus far, together with the results of the final probe (mean/sd
probe), in which students were asked to compare two data-sets described in the
formal manner of a mean and a standard deviation of the mean.
Table 4.
Students’ action and reasoning for data
collection, data processing and data-set
comparison after instruction.
Classification
of student
reasoning from
all previous
probes
Consistent point
paradigm reasoning
Inconsistent paradigm
reasoning
Consistent set
paradigm reasoning
Not
codeable
Total
Point or set action for
mean/sd probe
Point
Set
Not
paradigm
paradigm
codeable
action
action
3
6
0
(4%)
(9%)
(0%)
11
17
1
(17%)
(24%)
(1%)
14
16
0
(20%)
(23%)
(0%)
1
1
0
(1%)
(1%)
(0%)
29
40
1
(42%)
(57%)
(1%)
Total
9
(13%)
29
(42%)
30
(43%)
2
(3%)
70
(100%)
From Table 4 it is clear that although more than half the students (57%) carried
out an action associated with the set paradigm, fewer than half of this group (23% of
the sample) provided a reason that was also consistent with this paradigm. In other
words 33% of the students (57%? 23%? 1%) appear to have used the correct set
paradigm action either by rote or in an ad hoc way. In summary, only a quarter of
all the students (100%? 42%? 33%) can be regarded as having completely embraced
the set paradigm.
Conclusions
In terms of the constructs of the point and set paradigms, the purpose of
laboratory instruction can be regarded as attempting to shift students’ actions and
reasoning away from the point paradigm to those commensurate with the set
paradigm. Both the present study and the previous work (Buffler et al., 1998)
strongly suggest that students come from school firmly located within the point
paradigm, and that any set paradigm actions (such as calculating a mean) are most
often a rote response. Even after a six month laboratory course, only about one
quarter of the students seem to have reached the required instructional goals. The
present probes and the analysis framework appear to offer useful research tools that
can be used to evaluate the effectiveness of any laboratory curriculum that aims to
address procedural understanding in the context of experimentation.
References
Allie, S. and Buffler, A. (1998). A course in tools and procedures for Physics 1. American
Journal of Physics, 66 (7), 612-623.
Allie, S., Buffler, A., Kaunda, L., Campbell, B. and Lubben, F. (1998). First year physics
students’ perceptions of the quality of experimental measurements. International Journal
of Science Education, 20 (4), 447-459.
Buffler, A., Allie, S., Campbell, B. and Lubben, F. (1998) The role of laboratory experience
at school on the procedural understanding of pre-first year science students at UCT. In N.
A. Ogude and C. Bohlmann, Eds., Proceedings of the 6th Annual Meeting of the Southern
African Association for Research in Mathematics and Science Education, (pp. 495-502).
Pretoria, South Africa: University of South Africa Press.
Coelho, S. M. and Séré, M.-G. (1998). Pupils’ reasoning and practice during hands-on
activities in the measurement phase. Research in Science and Technological Education,
16 (1), 70-96.
Evangelinos, D., Valassiades, O. and Psillos, D. (1999). Undergraduate students’ views about
the approximate nature of measurement results. In M. Komorek et al., Eds., Proceedings
of the Second International Conference of the European Science Education Research
Association Volume 1, (pp. 208-210). Kiel, Germany: IPN.
Lubben, F. and Millar, R. (1996). Children’s ideas about the reliability of experimental data.
International Journal of Science Education, 18 (8), 955-968.
Séré, M.-G., Journeaux, R. and Larcher, C. (1993). Learning the statistical analysis of
measurement error. International Journal of Science Education, 15 (4), 427-438.
Acknowledgements
The research reported in this paper was supported by the British Council (DFID), the
University of Cape Town and the University of York. We also thank Indresan
Govender and Fiona Gibbons for their assistance.