Michael Luke NJDOE
609-984-9637
michael.luke@doe.state.nj.us
Topics
 Resources
 Premises
 Review of Preliminary Statistics
 Constructed Response Scoring Observations
 Constructed Response Sample Items
 Questions
Resources
 American Diploma Project (ADP) End-of-Course Exams:




2010 Annual Report
www.achieve.org/2010ADPAnnualReport
An Exploratory Teacher Survey Related to the ADP
Algebra I End-of-Course Exam
www.pearsonaccess.com
www.achieve.org/AlgebraITestOverview
Achieve Simplify Document
www.achieve.org/files/ADPSimplify.pdf
Premises of Presentation
 Student Performance on Constructed Response Items (both
2 and 4 point items) is well below student performance on
Multiple Choice Items
 A review of ADP Constructed Response Item Expectations
and Scoring could enhance student achievement on the
ADP Algebra I assessment
 “The survey results revealed that teachers underestimated
the rigor of the constructed-response items on the exam
and were not always clear on the kinds of items that are
being used to assess certain Algebra I benchmarks.”
Algebra I Test Blueprint
 40 Multiple Choice (MC) Items (@ 1 point each= 40




points)
5 Short Answer (SA) Items (@ 2 points each= 10 points)
2 Extended Response (ER) Items (@ 4 points each= 8
points)
58 Raw Points- Maximum Score
18/58= 31% of score points from Open Ended
(Constructed Response) Items
2009 Raw Cut Scores
 Four Proficiency Levels- Below Basic, Basic, Proficient,
Advanced (see letter from Acting Commissioner Hendricks
dated September 24, 2010)
 Raw Cut Scores 2009
 Basic 18 (387, scale 300-850)
 Proficient 26 (44.8%) (450)
 Advanced 42 (575)
NJ/Consortium Performance
 C 2010 A 4.8%, P 24.1%, B 24.8%, BB 46.4%
 P or A Performance Requires 26 Points Minimum (C- 2009)
 NJ Raw Mean Score 2009 18.43
 NJ Mean Raw Points from MC Items 2009 15.74 (39.35%)
 NJ Mean Raw Points from OE Items 2009 2.69 (14.9%)
Omission Rates (C- 2010)
 Percent of Students that left All Constructed Response Items
Blank- 2.0%
 Percent of Students that Left All Extended Response Items
blank- 7.8%
 Percent of Student that Left all Short Answer Items Blank2.4%
Constructed Response Results(C2010)
 Percent of Students that Scored 0 Points on All Constructed
Response Items- 23.4% (23.4-2=21.4%)
 Percent of Students that Scored 0 Points on All Extended
Response Items- 33.5%
 Percent of Students that Scored 0 Points on all Short
Answer Items- 42.0% (compare w/ bullet 2 above)
Review of Statistics
 Mean Raw Score (NJ 2009)- 18.43(26)
 Mean Raw Points from OE Items (NJ 2010)-2.69
 Omission Rates don’t fully account for bullet #2
 23.4% of examinees get 0 from Constructed Response
Items
Scoring Rubrics
 Paper #1- NJ ASK 8 Item, Holistic Scoring
 Paper #2- ADP Algebra I Release Item, SR- 2 point max item,




Analytic Scoring, Use of Value Points, Two key points to observe.
Dichotomous distribution of scores.
All answer spaces for Constructed Response (2 and 4 point
items) Items have grids
Highest score points are reserved for perfect papers (2.69)
Constructed Responses items are embedded in test parts and
are not identified as 2 or 4 point items
Paper #3- ADP Release Item, ER- 4 points max
Items Developed to Illustrate
Scoring Notes
The length of a rectangle is 8 inches longer than the width of
the rectangle. The area of the rectangle is 33 square
inches. Find the dimensions of the rectangle.
 Value points- how many?
Items Developed to Illustrate
Scoring Notes
Graph y=x²-2x-8. (N1.a.)
1. Could be asked to find vertex
2. Could be asked to graph using x- intercepts and vertex
3. Shape will account for a value point (graph 5-points)
4. Watch concavity
5. Graph should be a function
Items Developed to Illustrate
Scoring Notes
Graph the solution set of y≤2x+5. (L2.c.)
 Shading method should fully indicate a half plane
 Proper type of border line (shape and pattern)
Items Developed to Illustrate
Scoring Notes
Solve 4x²=49 (N2.b.)
 Also see ADP sample #1
 Show a solution strategy that accounts for all solutions, not
just principal square root- this will account for a value point
and in effect, a score point
 Solve |x-5|=26 (L2.b.)
Items Developed to Illustrate
Scoring Notes
Simplify √(50x²y³) (O2.d.)
 Must account for absolute value (if necessary) for full score
point value
See Remaining ADP
Release/Sample Items
Questions?
 609-984-9637
 michael.luke@doe.state.nj.us