AP European History
College
Comparability Study
A Presentation and Discussion at
2007
the 2007 AP Annual Conference,
Las Vegas Nevada, June 14
Paul Borysewicz
Educational Testing Service
The following PowerPoint presentation represents
data gathered from a comparability study
conducted in 2007. During a college
comparability study a portion of an AP Exam is
administered to college students and the freeresponse portion is scored during the AP reading.
The results of a comparability study is one factor
that is considered when setting AP grades in
order to ensure that the AP standards are
comparable with or higher than the performance
of college students who participated in the study.
2
AP European History, July 14, 2007
Why do we do comparability studies?
• To create a better fit between college-level
grading standards and AP standards
• To improve the validity of AP exams and
courses
3
AP European History, July 14, 2007
How is the study performed?
• College students are given a mini-version of the
exam (one FRQ and 20 MC questions)
• Their instructors grade the exams on the A-F
scale
• AP readers grade the college essays using the
same standards as regular AP essays
• Statisticians extrapolate an AP grade for each
college student and compare it to that student’s
college grade
4
AP European History, July 14, 2007
Who participated in the 2007 study for
APEH?
Nearly 700 students taking European history survey courses at 16 colleges:
University of South Carolina Upstate
University of Southern California
Oregon State University
University of Maryland
Baylor University
Elon University
Washington University in St. Louis
Michigan State University
Furman University
University of Louisville
Purdue University
University of South Florida
University of New Mexico
Duquesne University
Iowa State University
East Carolina University
5
AP European History, July 14, 2007
What were the results of the study?
• AP students at the 5-4 cut point scored higher than the
average score earned by a college student at the level
• AP students at the 4-3 cut point scored higher than the
average score earned by a college student at the B level
• AP students at the 3-2 cut point scored higher than the
average score earned by a college student at the C level
• AP students at the 2-1 cut point scored lower than the
average score earned by a college student at the D level
6
AP European History, July 14, 2007
How did the results affect scoring?
• The cut points for 5-4, 4-3, and 3-2 were
unchanged
• The cut point for 2-1 was raised to bring it
more in line with college-level performance
at the D level. Thus, there were more AP
grades of 1.
7
AP European History, July 14, 2007
What was the rationale for the decisions?
• The average college scores may have been lower
than AP scores because of factors unrelated to
overall ability (motivation and lack of a truly
representative sample) Hence, cut points were
not lowered in response to stronger performance
by AP students
• With expansion of AP, there seems to be a larger
percentage of poorly prepared or unmotivated
students taking the test. This is consistent with
an increased percentage of grades of 1.
8
AP European History, July 14, 2007
What are the implications of the study for the
future of APEH?
• The study indicated no serious discrepancy
between APEH course content and grading
standards and those of the college courses
in the survey
• The study is consistent with other evidence
indicating an increase in the number of
poorly prepared/unmotivated APEH
students
9
AP European History, July 14, 2007
What can you, as APEH teachers do?
• Keep doing what you’re doing!
• Pay close attention to the Course
Description and supporting materials
(especially as we move toward redesign of
the AP history courses)
• Work on developing students’ writing and
analytic skills
10
AP European History, July 14, 2007