BASIC TESTING TERMINOLOGY
•
•
•
•
Norm-referenced &
criterion referenced tests
Formative and summative tests
Objective & subjective tests
1
LECTURE’S OBJECTIVES
 Identify the different formats of tests found
 Distinguish different types of tests: norm-referenced
and criterion-referenced tests
 Compare and contrast formative and summative
tests
 Differentiate between objective and subjective tests
(Main reference - Brown, H. Douglas, 2004. Language
Assessment: Principles and classroom practices. )
2
NORM-REFERENCED
&
CRITERION REFERENCED
TESTS
3
NORM-REFERENCED TESTS
 To rank each student with respect to the achievement of
others in broad areas of knowledge.
 Normed using large groups of test takers. Compares one
taker to another. Measure achievement, predicts future
performance.
 Each individual is compared with other examinees and
assigned a score--usually expressed as a percentile, a
grade or equivalent score.
 Student achievement is reported for broad skill areas,
although some norm-referenced tests do report student
achievement in specific sub-areas.
4
NORM-REFERENCED TEST
 Measures broad skill areas sampled from a variety of
textbooks, syllabi, and the judgments of curriculum
experts.
 Each skill is, usually, tested by less than four items.
 Items vary in difficulty. Items are selected that
discriminate between high and low achievers.
• If too many people get a question correct, or too many
score well, then test questions are “thrown out” until they
achieve a normal curve again.
5
CRITERION-REFERENCED TEST
• Criterion-referenced tests, also called mastery tests,
compare a person's performance to a set of
objectives. Anyone who meets the criterion can get a
high score.
• Everyone knows what the benchmarks / objectives are
and can attain mastery to meet them.
• It is possible for ALL the test takers to achieve 100%
mastery.
• Measure a student against a specific set of knowledge
(criterion).
6
CRITERION-REFERENCED TEST
 To determine whether each student has achieved specific








skills or concepts.
To find out how much students know before instruction
begins and after it has finished.
Measures specific skills which make up a designated
curriculum.
These skills are identified by teachers and curriculum
experts.
Each skill is expressed as an instructional objective.
Each individual is compared with a preset standard for
acceptable achievement.
The performance of other examinees is irrelevant.
Each skill is tested by at least four items in order to obtain
an adequate sample of student performance and to
minimize the effect of guessing.
The items which test any given skill are parallel in
difficulty.
7
NORM & CRITERION REFERENCED TESTS
The following is adapted from: Popham, J. W. (1975). Educational
evaluation. Englewood Cliffs, New Jersey: Prentice-Hall, Inc.
Dimension
Purpose
Criterion-Referenced
Tests


Content


Norm-Referenced
Tests
To determine whether each
student has achieved specific
skills or concepts.
To find out how much
students know before
instruction begins and after it
has finished.

Measures specific skills which
make up a designated
curriculum. These skills are
identified by teachers and
curriculum experts.
Each skill is expressed as an
instructional objective.


To rank each student with
respect to the
achievement of others in
broad areas of knowledge.
To discriminate between high
and low achievers.
Measures broad skill areas
sampled from a variety of
textbooks, syllabi, and the
judgments of curriculum
experts.
8
NORM & CRITERION REFERENCED TESTS
Dimension
Item
Characteristics
Criterion-Referenced
Tests


Each skill is tested by
at least four items in
order to obtain an
adequate sample of
student performance
and to minimize the
effect of guessing.
The items which test
any given skill are
parallel in difficulty.
Norm-Referenced
Tests



Each skill is usually
tested by less than four
items.
Items vary in difficulty.
Items are selected that
discriminate between
high and low achievers.
9
NORM & CRITERION REFERENCED TESTS
Dimension
Score
Interpretation
Criterion-Referenced
Tests



Each individual is
compared with a preset
standard for acceptable
achievement. The
performance of other
examinees is irrelevant.
A student's score is
usually expressed as a
percentage.
Student achievement is
reported for individual
skills.
Norm-Referenced
Tests


Each individual is
compared with other
examinees and assigned a
score--usually expressed
as a percentile, a grade
equivalent score, or a
stanine.
Student achievement is
reported for broad skill
areas, although some
norm-referenced tests do
report student achievement
for individual skills.
10
NORM & CRITERION REFERENCED TESTS
Uses of Test Results for Teachers


Two main ways that test results can be used by teachers:
•
For revising instruction for entire class.
•
For developing intervention strategies for individual students.
Standardized test results have not typically been used to aid
teachers in making instructional decisions.

Data-driven decision making takes some practice and experience
for classroom teachers.
11
COMPARING NORM &
CRITERION-REFERENCED TESTS
• Norm-referenced
–
–
–
–
General ability
Range of ability
Large groups
Compares people to
people-comparison
groups
– Selecting top
candidates
• Criterion-referenced
–
–
–
–
–
–
Mastery
Basic skills
Prerequisites
Affective
Psychomotor
Grouping for instruction
12
COMMON CHARACTERISTICS
OF NRT & CRT
*Require a relevant and representative sample of
test items
*Require specification of the achievement domain to
be measured
*Use the same type of test items
*Use the same rules for item writing
*Judged by the same qualities (validity and reliability)
*Useful in educational measurement
13
ADVANTAGES AND
DISADVANTAGES OF NRT
Advantages：
 They easy for instructors to use
 They work well in situations requiring rigid differentiation among
students
 They are generally appropriate in large courses
Disadvantages：
 An individual's grade is determined not only by his/her
achievements, but also by the achievements of others.
 no indication of prerequisite knowledge for more advanced
material has been mastered
 less appropriate for measuring affective and psychomotor
objectives
 encourages competition and comparison scores
14
ADVANTAGES AND
DISADVANTAGES OF CRT
Advantages：
 Students are not competing with each other
 Students are thus more likely to actively help each other learn.
 A student's grade is not influenced by the caliber of the class.
Disadvantages：
 It is difficult to set a reasonable standard for students
 Most experienced faculty set criteria based on their knowledge of
how students usually perform
 Criterion-referenced systems often become fairly similar to normreferenced systems.
 absolute standards difficult to set in some areas
 standards tend to be arbitrary
 not appropriate comparison when others are valuable
15
FORMATIVE
&
SUMMATIVE
TESTS
16
THE GARDEN ANALOGY
If we think of our children as plants …
 Summative assessment of the plants is the process of
simply measuring them. It might be interesting to compare
and analyze measurements but, in themselves, these do
not affect the growth of the plants.
 Formative assessment, on the other hand, is the equivalent
of feeding and watering the plants appropriate to their
needs - directly affecting their growth.
 Formative and summative assessment are interconnected.
They seldom stand alone in construction or effect.
17
FORMATIVE ASSESSMENT
 Assessment for learning
 Taken at varying intervals throughout a course to provide
information and feedback that will help improve
 the quality of student learning
 the quality of the course itself
 The purpose is:
 To promote further improvement of student learning
during the learning process
 To involve students in the ongoing assessment of their
own achievement
 Provides information on what an individual student needs
 To practice
 To have re-taught
 To learn next
18
KEY ELEMENTS OF FORMATIVE
ASSESSMENT
1. The identification by teachers & learners of learning
2.
3.
4.
5.
goals, intentions or outcomes and criteria for
achieving these.
Rich conversations between teachers & students that
continually build and go deeper.
The provision of effective, timely feedback to enable
students to advance their learning.
The active involvement of students in their own
learning.
Teachers responding to identified learning needs and
strengths by modifying their teaching approach(es).
Black & Wiliam, 1998
19
BENEFITS OF FORMATIVE
ASSESSMENT FOR TEACHERS
(Boston, 2002)
 Teachers are able to determine what standards students
already know and to what degree.
 Teachers can decide what minor modifications or major
changes in instruction they need to makes so that all
students can succeed in upcoming instruction and on
subsequent assessments.
 Teachers can create appropriate lessons and activities
for groups of learners or individual students.
 Teaching can inform students about their current
progress in order to help them set goals for improvement.
20
BENEFITS OF FORMATIVE
ASSESSMENTS FOR STUDENTS
 Students are more motivated to learn.
 Students take responsibility for their own learning.
 Students become users of assessment.
 Students learn valuable lifelong skills such as self-
evaluation, self-assessment, and goal setting.
 Student achievement can improve from 21-41 percentile
points.
(marzano 2003; stiggens et. al, 2006)
21
SUMMATIVE ASSESSMENT
 Assessment of learning
 Generally taken by students at the end of a unit
or semester to demonstrate the "sum" of what
they have or have not learned.
 Summative assessment methods are the most
traditional way of evaluating student work.
 "Good summative assessments--tests and other
graded evaluations--must be demonstrably
reliable, valid, and free of bias" (Angelo and
Cross, 1993).
22
POSSIBLE ASSESSMENT METHODS
Formative Assessment includes







Questions
Classroom Discussions
Learning Activities
Feedback
Conferences
Interviews
Student Self-Assessment
Summative Assessment
 Selected Response
 Multiple Choice
 True/False
 Matching
 Fill-in
 Extended Written Response
 Performance Assessment
23
BALANCED CLASSROOM ASSESSMENT SYSTEM
FORMATIVE ASSESSMENT
A process used by teachers
and students during instruction
that provides feedback to
adjust ongoing teaching and
learning to help students
improve their achievement of
intended instructional
outcomes.
SUMMATIVE ASSESSMENT
A tool used after instruction
to measure student
achievement which provides
evidence of student
competence or program
effectiveness.
students are evaluated upon
completion of the work and
the focus is on the final
product.
24
COMPARISON OF ASSESSMENTS
FORMATIVE
•Occurs During Instruction
•Not Graded
•Process
•Descriptive Feedback
•Continuous
SUMMATIVE
•Occurs at the end
•Graded
•Product
•Evaluative Feedback
•Periodic
•Sort students in rank order
25
COMPARISON OF ASSESSMENTS
A Fine Sieve
 Formative assessment informs both teachers and
students about student understanding at a point when
timely adjustments can be made.
 These adjustments help to ensure students achieve
targeted standards-based learning goals within a set
time frame.
A course sieve
 Summative assessments happen too far down the
learning path to provide information at the classroom
level and to make instructional adjustments and
interventions during the learning process
26
OBJECTIVES
&
SUBJECTIVES
TESTS
27
OBJECTIVE AND SUBJECTIVE
ASSESSMENT
Objective assessment
is a form of questioning which has a single correct answer.
Subjective assessment
is a form of questioning which may have more than one
current answer (or more than one way of expressing the
correct answer).
28
OBJECTIVE TEST
 Objective tests include multiple choice, true-false,




matching, and fill-in questions. They tend to focus more
on specific facts than on general ideas and concepts
Questions on a tests that only have one correct answer
Objective tests require far more careful preparation than
subjective tests
Objective examination can be part of formative
(diagnostic) and summative (final assessment) exams.
Most popular objective exam is Multiple Choice
Questions (MCQ).
(the method of scoring is the only factor that distinguishes an objective test from a
subjective test)
 Advantages of multiple choice question:
1. The ability to create a test item bank
2. Quick grading – can be easily computer scored
3. If written well, high reliability - only one possible
4.
5.
6.
7.
8.
answer
Objective grading
Wide coverage of content
Can be used for mass testing
Precision in providing information regarding
specific skills and abilities.
Students are familiar with the item type –
directions are easy to understand.
 Weaknesses of multiple choice question:
1. Difficult and time consuming to construct
2. Low validity
3. Mainly tests recognition knowledge and recall of facts.
4. Guessing may have considerable effect
5. Cheating may be facilitated
6. Sometimes skills and areas are tested because they
are testable than important
7. Places a high degree of dependence on student’s
reading ability and teacher’s writing ability.
8. It may limit beneficial washback.
9. This technique strictly limits what can be tested.
SUBJECTIVE TEST
 Subjective tests include essay, short-answer,
vocabulary, and take-home tests
 Questions on a test that have more than one correct
answer.
 Each examiner uses his own judgment in evaluating
performance and awarding marks.
SUBJECTIVE TEST
 Strengths:
Easy to set
2. High validity
3. Can assess affective and interpretive aspects of language skills
4. allow a candidate to express originality of thought
5. allow the examiner to assess the candidate's quality of written
expression.
 Weaknesses:
1.
Marking is time consuming
2.
Reliability is low
3.
Inter-rater as well as intra-rater variability are probable.
4.
Dependence on presentation.- good hand writing vs bad
handwriting
5. Question evasion - possible for the candidate to avoid questions
in areas of the curriculum in which they are weak.
1.
OBJECTIVE VS. SUBJECTIVE TEST
Objective
Subjective
 short answer
 long answer
 closed response
 open response
 mostly recognition,
 emphasis on




limited production
difficult to write well
quick and easy to
grade
reliable
workload “up front”
production
 relatively easy to write
 difficult to grade
 time-consuming
 inter-rater reliability
 not as reliable
 workload post test
34
REFERENCES
 Classroom Assessment: Basic Concepts. Formative vs.Summative
Assessments. Retrieved October 20, 2008 from
http://fcit.usf.edu/assessment/basic/basica.html
 Formative vs. Summative Evaluation. Retrieved October 20, 2008
from http.jan.ucc.nau.edu/edtech/etc/667/proposal/evaluation/
summative_vs_formative.htm
 Formative and Summative Assessment. Retrieved October 20, 2008
from http://www.krauseinnovationcenter.org/ewyl/modules/module63.html.

Classroom Assessment: Basic Concepts. Formative vs.Summative
Assessments. Retrieved October 24, 2008 from
http://fcit.usf.edu/assessment/basic/basica.html

Pawlas, G., Oliva, P. (2008) Supervision for Today’s Schools, Sixth
Edition. New York: John Wiley and Sons
35
 Arter, Judith, and Jay McTighe. Scoring Rubrics in the
Classroom. Thousand Oaks, CA: Corwin Press, INC.,
2001.
 Marzano, Robert J., Debra Pickering, and Jay McTighe.
Assessing Student Outcomes. Alexandria, VA:
Association for Supervision and Curriculum
Development, 1993.
 Schoenbach, Ruth, et al. Reading for Understanding, A Guide to
Improving Reading in Middle and High School Classrooms.
San Francisco, CA: Jossey-Bass, Inc., 1999.
36