Assessment – Test/Exam Construction Learning Session- February 15, 2013
How we arrived here:
• Began Sept. 2012 with Common Final Exam initiative
• Collected all Final Exams in Jan. 2012
• Reviewed Final Exams – summary included today
• Introduced Final Exam Analysis Tool
• During Analysis Tool training - realized PD in exam writing
never occurred
• Discovered initial final exam directive -'fill the period’
• Adjustments to the changes in exam design at the state and
national level (i.e. Keystone, AP, PSAT, etc.) never made
Assessment – Test/Exam Construction Learning Session- February 15, 2013
Next Steps:
• PD today
• Guidelines for exam construction
• Provide resources (pull out possibilities w/ sub coverage)
•
February/March time planned - calendar will be provided
• Deadline to receive revised exams for Semester 2
•
May 10, 2013
Assessment – Test/Exam Construction Learning Session- February 15, 2013
Math
%mc
%fr
Math
%mc
%fr
alg2a
58%
42%alg 2 h
67%
33%
cp precalc
52%
48%calc h
0%
100%
trig h
0%
100%Geo a
52%
48%
acc trig
0%
100%geo cp
46%
54%
prob/stat cp
46%
54%alg 1A
70%
30%
prob.stat h
52%
48%alg 2 cp
55%
45%
alg 1 cp
prob/stat
acc
50%
50%alg 1b
73%
27%
65%
35%adv top
54%
46%
ap calc ab
63%
37%cp trig
39%
61%
geo H
0%
100%
Assessment – Test/Exam Construction Learning Session- February 15, 2013
Science
CP PS
A CP
fore
H Physics
ap physics
cp chem Z
chemcomm
cpchem ms
cp enviro
H chem
ap chem
%MC
%FR
100%
87%
92%
97%
31%
91%
97%
91%
98%
93%
90%
Science
0%cp physics
13%disasters
8%ap bio
3%H Enviro
69%cp bio
9%H bio
3%h kines
9%cp kines
2%a enviro
7%cp chem sav
10%
%MC
%FR
95%
33%
0%
100%
99%
100%
99%
100%
100%
5%
67%
100%
0%
1%
0%
1%
0%
0%
94%
6%
Assessment – Test/Exam Construction Learning Session- February 15, 2013
WL
g1
g2
s1
s2
s3
s4
f1
f2
f3
%MC %FR %List
72%
2% 26%
67%
9% 24%
84%
1% 15%
74% 13% 14%
90%
1%
9%
87%
2% 11%
78%
4% 18%
79%
5% 16%
73%
7% 21%
SS
geopoli
westciv cp
us his h
global H
Global cp
global a
amex
us his a
us cp
psych cp
psych ap
ap euro
market
us ap
west h
soc cp
%mc
%fr
86%
99%
87%
77%
95%
93%
99%
98%
99%
99%
0%
96%
21%
1%
13%
23%
5%
7%
1%
2%
1%
1%
100%
4%
98%
100%
2%
0%
n/a
n/a
Assessment – Test/Exam Construction Learning Session- February 15, 2013
Purpose:
 Guide Hatboro-Horsham educators as common
assessments are designed.
 Give consistent structure for test/exam construction.
 Provide a research-based tool and resources for
summative practices.
 Many types of assessment are utilized foster and monitor
student learning.
 While this resource is focused on effective exam design,
the information may be used during the process of
revising unit tests, benchmarks, and other content-based
assessments.
 Vital is to change/align instructional practices to reflect
higher-order thinking and student ownership of learning!
Assessment – Test/Exam Construction Learning Session- February 15, 2013
Test/Exam Construction:
 Bloom’s Taxonomy
 Webb’s Depth of Knowledge
 National and State Standardized Assessments
 Keystone Exams
 Higher-Order Thinking
 Extending the Learning
 Rigorous Content (CCSS)
Assessment – Test/Exam Construction Learning Session- February 15, 2013
Test/Exam Construction – Types of Questions:
Multiple Choice
Constructed Response/Open-Ended/Essay Questions
Multiple Choice :
 easily scored
 useful for analyzing general trends in student learning
 provide insight regarding student understanding of content
and skills
Constructed Response:
 use a specifically-designed rubric for scoring
 provides deep insight into student understanding and
thought processes
 requires students to apply, synthesize, and extend learning
Assessment – Test/Exam Construction Learning Session- February 15, 2013
Guidelines for Test/Exam Construction:
 Multiple-choice questions: core content and skills based on
common core standards and course curriculum
 No more than 60 multiple-choice questions on a test
 At least 30% of the multiple-choice questions are at the
application/analyze/evaluate levels of Bloom's Taxonomy
 Minimize recall/knowledge level questions
 Construct questions using the “Guidelines for Developing Test
Items”
Assessment – Test/Exam Construction Learning Session- February 15, 2013
Guidelines for Test/Exam Construction:
 Constructed response/open ended questions/essay
questions are at the application/analyze/evaluate levels of
Bloom's Taxonomy
 Constructed response/open ended questions/essay
questions are constructed using the “Guidelines for
Developing Constructed Response Items”
 Weighting: no more than 70% of test grade from multiple
choice/short response items; at least 30% of exam grade
from constructed response/open-ended questions/essay
questions
Assessment – Test/Exam Construction Learning Session- February 15, 2013
Assessment – Test/Exam Construction Learning Session- February 15, 2013
Checklist for Creating Common Assessments

Does each item or task require students to demonstrate the performance described
in the specific learning outcome it measures (relevance)?

Does each item present a clear and definite task to be performed (clarity)?

Is each item or task presented in simple, readable language and free from excessive
verbiage (conciseness)?

Does each item provide an appropriate challenge (ideal difficulty)?

Does each item have an answer that would be agreed upon by experts
(correctness)?

Is each item or task free from technical errors and irrelevant clues (technical
soundness)?

Is each test item free from cultural bias?
Assessment – Test/Exam Construction Learning Session- February 15, 2013
Keystone Exams – Multiple Choice and Constructed Response:
 Literature:http://static.pdesas.org/Content/Documen
ts/Keystone_Literature_Feb_2011_Item_and_Scoring
_Sampler.pdf
 Algebra_1:http://static.pdesas.org/Content/Docume
nts/Keystone_Algebra_I_Feb_2011_Item_and_Scorin
g_Sampler.pdf
 Biology:http://static.pdesas.org/Content/Documents
/Keystone_Biology_Feb_2011_Item_and_Scoring_Sa
mpler.pdf
Assessment – Test/Exam Construction Learning Session- February 15, 2013