Unit 1 Concepts of Testing, Measurement and Evaluation Introduction As a teacher-learner, you should be familiar with the process of assessment to which pupils are subjected from time to time. In this module, we shall discuss the subject of assessment in great detail. In so doing, we hope you will acquire both the knowledge and skills to enable you to assess pupils in your subject better. In this first unit, we shall try to establish the extent and content of the subject matter by examining some of the key concepts. You will need to understand the concepts of testing, measurement and evaluation in order to understand the content of this module. Quite often, teachers use these terms interchangeably because they all occur in a single process of assessment, but they are different. Domasi College of Education Objectives By the end of this unit, you should be able to: define the terms testing, measurement and evaluation distinguish between testing, measurement and evaluation in the assessment process distinguish between qualitative and quantitative variables associate attributes with appropriate measurement units. Key Words You will find the following key words or phrases in this unit. Watch out for these and make sure you understand what they mean and how they are used in the unit. test measurement evaluation qualitative formative evaluation summative evaluation attributes variable quantitative variable qualitative variable scales of measurement measurement unit nominal scale ordinal scale interval scale ratio scale Page 1 - 10 Education —Module 8, Introduction to Testing, Measurement and Evaluation Domasi College of Education Definitions of Key Concepts In this section we shall explain some of the key concepts presented in this course. You will notice that this module introduces a number of new terms. Understanding these terms is critical to understanding the course. It may therefore be necessary for you to develop the habit of listing new terms and their meanings as you study the course. Such a practice may help you understand and remember the new terms more easily. We shall now look at the terms test, measurement and evaluation. These terms are often easily confused because they all happen in a single process of assessment. For example, a teacher may ask a series of questions on a subject matter to which pupils may respond. The pupils’ responses are then scored by giving points to correct answers. On the basis of the pupils’ scores, the teacher may then conclude whether the pupils are making good progress or not. In this example of an assessment process a series of questions on the subject matter constitute a test scoring the responses by giving points to right responses is measurement concluding whether pupils are making good learning progress is evaluation. Let us now elaborate on the meaning of each of these terms. Test A test a systematic procedure for observing and describing one or more characteristics of a person with the aid of either a numerical scale or category system (Nitko, 1983). All systematic procedures used in a school to assess child behaviour, such as teacher observations, questionnaires, interviews, class projects, term papers or assignments and paper and pencil examinations, are some forms of tests. You will recall that in a teaching–learning process, each time tests are used they yield measurements, which describe behaviour numerically (using a numerical scale) or categorically (using a category system such as letter grades in the Junior Certificate Examination (JCE) or point scale in the Malawi School Certificate Examination (MSCE) and college assessments). Concepts of Testing, Measurement and Evaluation Page 1 - 11 Domasi College of Education Measurement Measurement is the process of assigning numbers to individuals or their characteristics according to specified rules. An example of a specified rule is assigning 1 to a right answer and 0 to a wrong answer. You may agree that measurement is a quantitative method of describing the attributes or characteristics of persons, which have been observed through testing. In order to come up with measurements, one needs to identify: the person to be measured the attributes or characteristics to be measured in that person the procedure or rule to be used to assign numbers to those characteristics. When measurements are used to describe attributes, one should ensure that the resulting scores represent the realworld relationship regarding the attribute being measured. For example, if you were to describe the heights of Mary and John in your classroom, your resulting measurements should agree with what people who see the two pupils think about their heights. If the results of measurements show that Mary is taller than John when real-world observation considers it differently, people will doubt the accuracy of the measurements. Similarly, if in a classroom testing one pupil performs outstandingly when her or his general performance has always been below average, it may cast doubt on the accuracy of measurements reported in the test. Self Evaluation — Activity 1-a Having gone through this section, how would you describe the relationship between testing and measurement? Possible answers to this activity are at the end of this unit. Page 1 - 12 Education —Module 8, Introduction to Testing, Measurement and Evaluation Domasi College of Education Evaluation Evaluation is the process of making a value judgement about the worth of someone or something being judged? Evaluation involves judgement, which may or may not be based on a test. When evaluation is based on a test, it uses measurements by comparing the numbers or categories to judge the worth of someone with regard to the characteristics being measured. When evaluation occurs in the absence of measurements or testing, it solely depends on what the evaluator thinks about the attribute or the person being judged. In both cases, evaluation depends on the experience and knowledge of the evaluator. You can agree that almost all evaluations are influenced by subjectivity and bias. There are two forms of evaluation which are used in curriculum development and instruction. These are formative evaluation summative evaluation. Formative Evaluation Formative evaluation monitors the instructional process to determine whether learning is taking place. It provides feedback to both teachers and students about how well things are going. Such feedback is used to modify instructional methods or materials and to facilitate learning if things have not been going well. Summative Evaluation A summative evaluation is conducted at the end of an instructional segment to determine if learning is complete and if the learner can move to the next segment of instruction. The major role of summative evaluation is to determine the status of achievement at the end of an instructional segment and determine how well things went. According to Gronlund and Linn (1990), summative evaluation describes judgements about an already completed programme, procedure or product. It may consider whether the stated objectives were achieved, or whether a particular method works, or under which conditions or under what degree of implementation. Concepts of Testing, Measurement and Evaluation Page 1 - 13 Domasi College of Education Assessment Task 1-b List all assessment activities which your school conducts, and categorise them as formative and summative evaluation procedures. You must submit this task together with Assessment Tasks 2-a, 3-c and 3-d to your instructor for feedback. This will be Assignment 1. Relationship Between Testing, Measurement and Evaluation You may now agree that there is a close relationship between the three concepts we have discussed so far. Evaluation seems to be a more comprehensive and inclusive term than testing and measurement. As we have seen, testing is just one type of measurement which represent gathering information through a series of questions. Tests are a subset of the quantitative tools or techniques used in evaluations. In a way, tests and measurements are necessary to increase the objectivity of evaluations. Figure 1.1 illustrates the relationship we have described in this section. Evaluation Measurement such Non-measurement as testing observation such as informal Value judgement such as Good learning progress Fig. 1.1. Relationships of Evaluation, Measurement, etc. Source: Gronlund and Linn, 1990 page 6 Page 1 - 14 Education —Module 8, Introduction to Testing, Measurement and Evaluation Domasi College of Education Other Relevant Concepts or Terms You need to learn other terms in order to understand testing, measurement and evaluation better. Below are some key definitions. Attribute or characteristic refer to a quality regarded as natural or typical of someone, such as height, weight, eye colour and so on. Variable refers to a phenomenon which varies from one trial to another during observation. Quantitative variable is an attribute or characteristic whose observations vary in magnitude from trial to trial, such as family size, class size, mortality rate or cost of living. Qualitative variable is an attribute or characteristic whose observations vary in kind but not in degree, such as sex, prison status, political affiliation, marital status and religious affiliation. Scales of Measurement There are four scales of measurement scales used in research and evaluation to describe variables. You will notice that these measurement scales express quantities of various variables in appropriate measurement units. There are many different measurement units: for example, distance is expressed in millimetres, centimetres, metres and kilometres or inches, feet, yards, and miles. Weight is described in grams and kilograms or ounce and pounds. Intelligence is expressed in intelligence quotient while temperature uses degrees Celsius or degrees Fahrenheit. Tests use points, scores or grades. The four measurement scales discussed here are nominal scale ordinal scale interval scale ratio scale Let us look at each of these measurement scales in more details. Concepts of Testing, Measurement and Evaluation Page 1 - 15 Domasi College of Education Nominal Scale Nominal scale refers to the process of grouping objects into classes while ensuring their equivalence with regard to some attribute or property. Each class relates to a qualitative category that is either named or numbered. The names or numerals are given for identification purposes. For example, sex may be coded 1 for male and for female. This scale is called nominal because it is a way of giving names to measurements. Ordinal Scale An ordinal scale is the type of measurement that describes rank ordering of things from low to high. Apart from incorporating the features of the nominal scale, ordinal measurements rank or order observations by degree. The numerical values used in an ordinal scale of measurement represent differing amounts of the characteristics being measured such as athletic achievement. Similarly, some tests demand a rank ordering of things, as may be the case with some questionnaires. Interval Scale An interval scale incorporates all the features of the ordinal scale. In addition it specifies distances between its levels, and its zero point is arbitrary and does not correspond to the absence of the characteristics being measured. For example 0 0C is not a total absence of heat or temperature, and the difference between 50 0C and 60 0C is equal to the difference between 90 0C and 100 0C. But 1000C is not twice 50 0C in terms of heat molecular motion (Wiersma & Jurs: 1990). Ratio Scale A ratio scale makes ratio type comparisons possible because its zero point indicates an absence of the property being measured. For example, zero height is no height and two kilometres long is one half of four kilometres. Just like the interval scale, in the ratio scale equal differences between the numbers assigned in measurement reflect equal differences in the amount of the property possessed by things being measured. The scale is called a ratio scale because its ratios are meaningful. With ratio scale, it is possible to consider proportions or fractions of quantities of things. We can talk of enrolment of Chichiri Secondary School of 1200 is ten times that of Songani Community Day Secondary School. Page 1 - 16 Education —Module 8, Introduction to Testing, Measurement and Evaluation Domasi College of Education Hands On Activity 1-c From your regular observation, identify appropriate measurement scales for the following variables: 1. Societies and clubs numbering in the school register 2. Sizes of classrooms 3. Numbering of players in a volleyball team 4. Shoe sizes for teachers in the school 5. Salaries of teachers in the school 6. Colour of shoes for girls in the school 7. Years of Christian calendar 8. Ranks of pupils in a test 9. Ranking of one’s preferences in a questionnaire 10. Reading of temperatures on a thermometer Answers to this activity are at the end of this unit. Reading Assignments At your own time read the following in the library: Hopkins, K.D. Stanley J.C., & Hopkins B.R. (1990). Educational and Psychological Measurement and Evaluation. Needham Heights, Massachusetts: Allyn and Bacon (pp 1-7) Wiersma, W. & Jurs, S.G. (1990). Educational Measurement and Testing. Needham Heights, Massachusetts: AALyn and Bacon (pp7-13) Ott, R.L. Rexroat, C, Larson, R., & Mendenhall W. (1992). Statistics: Tool for the Social Sciences. Boston: PWS – Kent Publishing Company (pp18-42) Concepts of Testing, Measurement and Evaluation Page 1 - 17 Domasi College of Education Reflection Imagine what could happen in assessment if schools did not use measurement when evaluating pupils who complete instructional segments. Summary In this unit, we have attempted to define the key concepts which should be understood when studying testing, measurement and evaluation. This information will help you understand the other issues covered in this module. This unit has elaborated in particular on the topics of evaluation and measurement, which are necessary components in testing. In the next unit, we shall examine reasons why the education process includes testing and evaluation. Unit 1 Test In your own words, define the following terms: (i) test (ii) formative evaluation (iii) ordinal measurement (iv) attribute (v) qualitative variable. Answers to this unit test are at the end of this unit. Page 1 - 18 Education —Module 8, Introduction to Testing, Measurement and Evaluation Domasi College of Education Answers to Unit 1 Activities Answers to Self Evaluation Activity 1a Relationship between testing and measurement Testing is a subset of measurement. Testing is a necessary condition for measurement because measurement cannot be obtained without systematic observation (testing) of the characteristic to be measured. Testing procedure helps to identify rules or tools for measurement. Assessment Task 1-b Assessment activities You must submit this task together with assessment tasks 2-a, 3-c and 3-d to your instructor for feedback. This will be Assignment 1. Answers to Hands on Activity 1-c 1. Nominal scale 2. Ratio scale 3. Nominal scale 4. Ratio scale 5. Ratio scale 6. Nominal scale 7. Interval scale 8. Ordinal scale 9. Ordinal scale 10. Interval scale Concepts of Testing, Measurement and Evaluation Page 1 - 19 Domasi College of Education Answers to Unit 1 Test 1. Test: a series of questions on a subject matter or a systematic procedure of observing attributes. 2. Formative evaluation: an evaluation aimed at assessing how well things are going. 3. Ordinal measurement: a measurement scale for things that must be ranked, such as positions of pupils in class tests results. 4. Attribute: a characteristic which is typical of someone, such as height. 5. Qualitative variable: an attribute which varies in kind and not in magnitude, such as marital status. Page 1 - 20 Education —Module 8, Introduction to Testing, Measurement and Evaluation