MODULE 6
Administering,
Analyzing, and
Improving
Tests
Introduction
One of the most important functions of a teacher is to assess the performance
of the students. This is very complicated task because you will consider many
activities such as the timing of the assessment process, the format of the
assessment tools and the duration of the assessment procedures.
After designing the assessment tools, package the test, administer the test to
the students, check the test papers, score and then record them. Return the test
papers and then give feedback to the students regarding the result of the test.
Packaging and Reproducing Test Items
Assuming that you have already assembled the test, you write the instructional
objectives, prepare the table of specification, and write test items that match with
the instructional objectives, the next thing to do is to package the test and
reproduce it.
1. Put the items with the same format together.
2. Arrange the test items from easy to difficult.
3. Give proper spacing for each item for easy reading.
4. Keep questions and options in the same page.
5. Place the illustrations near the options.
6. Check the key answer.
7. Check the direction of the test.
8. Provide space for name, date and score.
9. Proofread the test.
10. Reproduce the test.
Administering the Examination
After constructing the test items and putting them in order, the next step is to
administer the test to the students. The administration procedures greatly affect
the performance of the students in the test. The test administration does not simply
mean giving the test questions to the students and collecting the test papers after
the given time.
Guidelines Before Administering Examinations
1. Try to induce positive test-taking attitude.
2. Inform the students about the purpose of the test.
3. Give oral directions as early as possible before distributing the tests.
4. Give test-taking hints about guessing, skipping, and the like, are strictly
prohibited.
5. Inform the students about the length of time allowed for the test. If
possible, write on the board the time in which they must be finished with
answering the test. Give the students a warning before the end of the time
limit.
6. Tell the students how to signal or call your attention if they have a question.
7. Tell the students what to do with their papers when they are done answering
the test (how papers are to be collected).
8. Tell the students what to do when they are done with the test, particularly
if they are to go on to another activity (also write these directions on the
chalkboard so they can refer to them).
9. Rotate the method of distributing papers so you don’t always start from the
left or the front row.
10. Make sure the room is well lighted and has a comfortable temperature.
11. Remind students to put their names on their papers (and where to do so).
12. If the test has more than one page, have each student checked to see that
all pages are there.
Guidelines During the Examination
1. Do not give instructions or avoid talking while examination is going on to
minimize interruptions and distractions.
2. Avoid giving hints.
3. Monitor to check student progress and discourage cheating.
4. Give time warnings if students are not pacing their work appropriately.
5. Make a note of any questions students ask during the test so that items can
be revised for fortune use.
6. Test papers must be collected uniformly to save time and to avoid test papers
to be misplaced.
Guidelines After the Examination
After the examination, the next activity that the teacher needs to do is to
score the test papers, record the result of the examination, return the test
papers and last to discuss the test items in the class so that you can analyze and
improve the test items for future use.
1. Grade the papers (and add comments if you can); do test analysis after scoring
and before returning papers to students if at all possible. If it is impossible to
do your test analysis before returning the papers, be sure to do it at another
time. It is important to do both the evaluation of your students and the
improvement of your tests.
2. If you are recording grades or scores, record them in pencil in your class record
before returning the papers. If there are errors/adjustments in grading, they
(grades) are easier to change when recorded in pencil.
3. Return papers in a timely manner.
4. Discuss test items with the students. If students have questions, agree to look
over their papers again, as well as the papers of others who have the same
questions. It is usually better not to agree to make changes in grades on the spur
of the moment while discussing the tests with the students but to give yourself
time to consider what action you want to take. The test analysis may have already
alerted you to a problem with a particular question that is common to several
students, and you may already have made a decision regarding that question (to
disregard the question and reduce the highest possible score accordingly, to give
all students credit for that question, among others).
Analyzing the Test
After administering and scoring the test, the teacher should also analyze the
quality of each item in the test. Through this you can identify the item that is good,
item that needs improvement or items to be removed from the test. But when do we
consider that the test is good? How do we evaluate the quality of each item in the
test? Why is it necessary to evaluate each item in the test? Lewis Aiken (1997) an
author of psychological and educational measurement pointed out that a
“postmortem” is just as necessary in classroom assessment as it is in medicine.
The technique to help teachers determine the quality of a test item is known as
item analysis. One of the purposes of item analysis is to improve the quality of the
assessment tools. Through this process, we can identify the item that is to be
retained, revised or rejected and also the content of the lesson that is mastered or
not.
There are two kinds of item analysis, quantitative item analysis and qualitative
item analysis (Kubiszyn and Borich, 2007)
Item Analysis
Item analysis is a process of examining the student’s response to individual item
in the test. It consists of different procedures for assessing the quality of the test
items given to the students. Through the use of item analysis, we can identify which
of the given are good and defective test items. Good items are to be retained and
defective items are to be improved, to be revised or to be rejected.
Uses of Item Analysis
1. Item analysis data provide a basis for efficient class discussion of the test
results.
2. Item analysis data provide a basis for remedial work.
3. Item analysis data provide a basis for general improvement of classroom
instruction.
4. Item analysis data provide a basis for increased skills in test construction.
5. Item analysis procedures provide a basis for constructing test bank.
Types of Quantitative Item Analysis
There are three common types of qualitative item analysis which provide
teachers with three different types of information about individual test items.
1. Difficulty Index
It refers to the proportion of the number of students in the upper and
lower groups who answered an item correctly. The larger the proportion, the more
students, who have learned the subject is measured by the item. To compute the
difficulty index of an item, use the formula:
Dy = _n_
N
,
where
Dy = difficulty index
n = number of the students selecting item correctly in the upper group and
in the lower group
N = total number of students who answered the test.
Level of Difficulty
To determine the level of difficulty of an item, find first the difficulty
index using the formula and identify the level of difficulty using the range given
below.
Level of Difficulty of an Item
Index Range
Difficulty Level
0.00 – 0.20
Very Difficult
0.21 – 0.40
Difficult
0.41 – 0.60
Average/Moderately Difficult
0.61 – 0.80
Easy
0.81 – 1.00
Very Easy
The higher the value of the index of difficulty, the easier the item is.
Hence, more students got the correct answer and more students mastered the
content measured by the item.
2. Discrimination Index
The power of the item to discriminate the students between those who
scored high and those who scored low in the overall test. In other words, it is the
power of the item to discriminate the students who know the lesson and those
who do not know the lesson.
It also refers to the number of students in the upper group who got an
item correctly minus the number of students in the lower group who got an item
correctly. Divide the difference by either the number of the students in the
upper group or the number of students in the lower group or get the higher
number if they are not equal.
Discrimination index is the basis of measuring the validity of an item. This
index can be interpreted as an indication of the extent to which overall knowledge
of the content area or mastery of the skills is related to the response on an item.
Types of Discrimination Index
1. Positive discrimination happens when more students in the upper group
got the item correctly than those students in the lower group.
2. Negative discrimination occurs when more students in the lower group
got the item correctly than the students in the upper group.
3. Zero discrimination happens when a number of students in the upper
group and lower group who answer the test correctly are equal, hence,
the test item cannot distinguish the students who performed in the
overall test and the students whose performance are very poor.
Level of Discrimination
Ebel and Frisbie (1986) as cited by Hetzel (1997) recommended the use
of Level of Discrimination of an Item for easier interpretation.
Index Range
Discrimination Level
0.19 and below Poor item, should be eliminated or need to be revised
0.20 – 0.29
Marginal item, needs some revision
0.30 – 0.39
Reasonably good item but possibly for improvement
0.40 and above
Very good item
Discrimination Index Formula
DI
= CUG - CLG , where
D
DI = discrimination index value
CUP = number of students selecting the correct answer in the upper group
CLG = number of students selecting the correct answer in the lower group
D = number of students in either the lower group or upper group
Note: Consider the higher number in case the sizes in upper group and lower group
are not equal.
Steps in Solving Difficulty Index and Discrimination Index
1. Arrange the scores from highest to lowest.
2. Separate the scores into upper group and lower group. There are different
methods to do this: (a) if a class consists of 30 students who takes an exam,
arrange their scores from highest to lowest, then divide them into two groups.
The highest score belongs to the upper group. The lowest score belongs to
the lower group. (b) Other literatures suggested to use 27%, 30%, or 33% of
the students for the upper group and lower group. However, in the Licensure
Examination for Teachers (LET), the test developers always used 27% of the
students who participated in the examination for the upper and lower groups.
3. Count the number of those who chose the alternatives in the upper and lower
group for each item and record the information using the template:
Options
A
B
C
D
E
Upper Group
Lower Group
Note: Put asterisk for the correct answer.
4. Compute the value of the difficulty index and the discrimination index and
also the analysis of each response in the distracters.
5. Make an analysis for each item.
Checklist for Discrimination Index
It is very important to determine whether the test item will be retained,
revised or rejected. Using the Discrimination Index we can identify the nonperforming question items; just always remember that they seldom indicate what is
the problem. Use the given checklist below:
Yes
No
1. Does the key discriminate positively?
2. Does the incorrect options discriminate negatively?
If the answers to questions 1 and 2 are both YES, retain the item.
If the answers to questions 1 and 2 are either Yes or No, revise the item.
If the answers to questions 1 and 2 are both NO, eliminate or reject the item.
3. Analysis of Response Options
Aside from identifying the difficulty index and discrimination index, another
way to evaluate the performance of the entire test item is through the analysis of
the response options. It is very important to examine the performance of each
option in a multiple-choice item. Through this, you can determine whether the
distracters or incorrect options are effective or attractive to those who do not know
the correct answer. The attractiveness of the incorrect options is determined when
more students in the lower group than in the upper group choose it. Analyzing the
incorrect options allows the teachers to improve the test items so that it can be
used again in the future.
Distracters Analysis
1. Distracter
Distracter is the term used for the incorrect options in the multiple-choice
type of test while the correct answer represents the key. It is very important
for the test writer to know if the distracters are effective or good distracters.
Using quantitative item analysis, we can determine if the options are good or if
the distracters are effective.
Item analysis can identify non-performing test items, but this item seldom
indicates the error or the problem in the given item. There are factors to be
considered why students failed to get the correct answer in the given question.
a. It is not taught in the class properly.
b. It is ambiguous.
c. The correct answer is not in the given options.
d. It has more than one correct answer.
e. It contains grammatical clues to mislead the students.
f. The student is not aware of the content.
g. The students were confused by the logic of the question because it has
double negatives.
h. The student failed to study the lesson.
2. Miskeyed Item
The test item is a potential miskey if there are more students from the
upper group who choose the incorrect options than the key.
3. Guessing Item
Students from the upper group have equal spread of choices among the
given alternatives. Students from the upper group guess their answers
because of the following reasons:
a. The content of the test is not discussed in the class or in the text.
b. The test item is very difficult.
c. The question is trivial.
4. Ambiguous Item
This happens when more students from the upper group choose equally
an incorrect option and the keyed answer.
Qualitative Item Analysis
Qualitative item analysis is a process in which the teacher or expert carefully
proofreads the test before it is administered, to check if there are typographical
errors, to avoid grammatical clues that may lead to giving away the correct answer,
and to ensure that the level of reading materials is appropriate. These procedures
can also include small group discussions on the quality of the examination and its
items, with examinees that have already took the test. According to Cohen, Swerdlik,
and Smith (1992) as cited by Zurawsski, students who took the examination are
asked to express verbally their experience in answering each item in the examination.
This procedure can help the teacher in determining whether the test takers
misunderstood a certain item, and it can help also in determining why they
misunderstood a certain item.
Improving Test Items
As presented in the introduction of this chapter, item analysis enables the
teachers to improve and enhance their skills in writing test items. To improve
multiple-choice test item, we shall consider the stem of the item, the distracters
and the key answer.
How to Improve the Test Item
Consider the following examples in analyzing the test item and some notes on
how to improve the item based from the results of item analysis.
Example 1. A class is composed of 40 students. Divide the group into two.
Option B is the correct answer. Based from the given data on the table, as a teacher,
what would you do with the test item?
Options
A
B*
C
D
E
Upper Group
3
10
4
0
3
Lower Group
4
4
8
0
4
1. Compute the difficulty index.
n = 14 + 4 = 14
N = 40
DF =
n_
N
DF =
14_
40
DF = 0.35 or 35%
2. Compute the discrimination index.
CUG = 10
CLG = 4
D = 20
DI = CUG – CLG
D
= 10 – 4
20
= 6__
20
= 0.30 or 30%
3. Make an analysis about the level of difficulty, discrimination and distracters.
a. Only 35% of the examinees got the answer correctly, hence, the item is
difficult.
b. More students from the upper group got the answer correctly, hence, the
it has a positive discrimination.
c. Retain options A, C, and E because most of the students who did not
perform well in the overall examination selected it. Those options attract
most students from the lower group.
4. Conclusion: Retain the test item but change option D, make it more realistic
to make it effective for the upper and lower groups. At least 5% of the
examinees choose the incorrect option.
Example 2. A class is composed of 50 students. Use 27% to get the upper and
the lower groups. Analyze the item given the following results. Option D is correct
answer. What will you do with the test item?
Options
A
B
C
D*
E
Upper Group (27%)
3
1
2
6
2
Lower Group (27%)
5
0
4
4
1
1. Compute the difficulty index.
n = 6 + 4 = 10
N = 28
DF = _ n _
N
DF = _ 10 _
28
DF = 0.36 or 36%
2. Compute the discrimination index.
CUG = 6
CLG = 4
D = 4
DI = CUG – CLG
D
DI = 6 – 4
14
DI = _2_
14
DI = 0.14 or 14%
3. Make an analysis.
a. Only 36% of the examinees got the answer correctly, hence, the item is
difficult.
b. More students from the upper group got the answer correctly, hence, it
has a positive discrimination.
c. Modify option B and E because more students from the upper group chose
them compare with the lower group, hence, they are not effective
distracters because most of the students who performed well in the
overall examination selected them as their answers.
d. Retain options A and C because most of the students who did not perform
well in the overall examination selected them as the correct answers.
Hence, options A and C are effective distracters.
4. Conclusion: Revised the item by modifying options B and E.
Example 3. A class is composed of 50 students. Use 27% to get the upper and the
lower groups. Analyze the item given the following results. Option E is the correct
answer. What will you do with the test item?
Options
A
B
C
D
E*
Upper Group (27%)
2
3
2
2
5
Lower Group (27%)
2
2
1
1
8
1. Compute the difficulty index.
n = 5 + 8 = 13
N = 28
DF = _n_
N
DF = _13_
28
DF = 0.46 or 46%
2. Compute the discrimination index.
CUG = 5
CLG = 8
D=4
DI = CUG – CLG
D
DI = _5 – 8_
14
DI = _-3_
14
DI = -0.21 or -21%
3. Make an analysis.
a. 46% of the students got the answer to test item correctly, hence, the test
item is moderately difficult.
b. More students from the lower group got the item correctly, therefore, it is
a negative discrimination. The discrimination index is -21%.
c. No need to analyze the distracters because the item discriminates negatively.
d. Modify all the distracters because they are not effective. Most of the
students in the upper group chose the incorrect options. The options are
effective if most of the students in the lower group chose the incorrect
options.
e. Conclusion: Reject the item because it has a negative discrimination index.
Example 4. Potential Miskeyed Item. Make an item analysis about the table
below. What will you do with the test that is a potential miskeyed item?
Options
A*
B
C
D
E
Upper Group
1
2
3
10
4
Lower Group
3
4
4
4
5
1. Compute the difficulty index.
n = 8 + 6 = 14
N = 39
DF = _n_
N
DF = _14_
39
DF = 0.36 or 36%
2. Compute the discrimination index.
CUG = 8
CLG = 6
D = 20
DI = CUG – CLG = _8 – 6_
D
20
DI = _2_
20
DI = 0.10 or 10%
3. Make an analysis.
a. Only 36% of the students got the answer to the test item correctly,
hence, the test item is difficult.
b. More students from the upper group got the item correctly, hence, it
discriminates positively. The discrimination index is 10%.
c. About equal numbers of top students went for option A and option E, this
implies that they could not tell which is the correct answer. The students
do not know the content of the test, thus, a reteach is needed.
4. Conclusion: Revise the test item because it is ambiguous.
Example 6. Guessing Item. Below is the result of an item analysis for a test item
with students’ answers mostly based on a guess. Are you going to reject, revise or
retain the test item?
Options
A
B
C*
D
E
Upper Group
4
3
4
3
6
Lower Group
3
4
3
4
5
1. Compute the difficulty index.
n=4+3=7
N = 39
DF = _n_
N
DF = _7_
39
= 0.18 or 18%
2. Compute the discrimination index.
CUG = 4
CLG = 3
D = 20
DI = CUG – CLG
D
DI = 4 – 3
20
DI = _1_
20
DI = 0.05 or 5%
3. Make an analysis.
a. Only 18% of the students got the answer to the test item correctly,
hence, the test item is very difficult.
b. More students from the upper group got the correct answer to the test
item; therefore, the test item is a positive discrimination. The
discrimination index is 5%.
c. Students respond about equally to all alternatives, an indication that
they are guessing.
Three possibilities why student guesses the answer on a test
item:
 the content of the test item has not yet been discussed
in the class because the test is designed in advance;
 test items were badly written that students have no idea
what the question is really all about; and
 test items were very difficult as shown from the
difficulty index and low discrimination index.
d. If the test item is well-written but too difficult, reteach the material
to the class.
4. Conclusion: Reject the item because it is very difficult and the
discrimination index is very poor, and options A and B are not effective
distracters.
Example 7. The table below shows an item analysis of a test item with
ineffective distracters. What can you conclude about the test item?
Options
Upper Group
Lower Group
A
5
6
B
3
4
C*
9
6
D
0
0
E
3
4
1. Compute the difficulty index.
n = 9 + 6 = 15
N = 40
DF = _n_
N
DF = _15_
40
DF = 0.38 or 38%
2. Compute the discrimination index.
CUG = 9
CLG = 6
D = 20
DI = CUG – CLG
D
DI = _9 – 6_
20
DI = _3_
20
DI = 0.15 or 15%
3. Make an analysis.
a. Only 38% of the students got the answer to the test item correctly
hence, the test item is difficult.
b. More students from the upper group answered the test item
correctly; as a result, the test got a positive discrimination. The
discrimination index is 15%.
c. Option A, B, and E are attractive and effective distracters.
d. Option D is ineffective; therefore, change it with more realistic one.
4. Conclusion: Revise the item by changing option D.