AP United States Government and Politics
College Comparability Study 2008
A Presentation and Discussion at the 2008 AP Annual
Conference
Seattle, WA, July 19, 2008
Ed Wagner
Educational Testing Service
Why do we do comparability studies?
• To create a better fit between college-level
grading standards and AP standards
• To improve the validity of AP exams and
courses
2
AP United States Government, July 19, 2008
How is the study performed?
• College students are given a mini-version of the exam (20
multiple choice questions and one free response question)
• Their instructors grade the exams on the A-F scale
• AP readers grade the college essays using the same
standards as regular AP essays
• Statisticians extrapolate an AP grade for each college
student and compare it to that student’s college grade
3
AP United States Government, July 19, 2008
AP US Government and Politics
Comparability Study: Sample Size
• 23 colleges/universities
• 1,171 college participants
4
AP United States Government, July 19, 2008
Who participated in the 2008 study for AP US
Government and Politics? Roughly 1,200 students
taking Intro to US Government at 23 colleges:
Temple University
University of Memphis
University of Georgia
University of New Orleans
University of Florida
University of Maryland
Old Dominion University
University of Louisville
City University of New York
University of Pittsburgh
Valparaiso University
Idaho State University
Hofstra University
Texas Southern University
University of Oklahoma
Wheaton College
Roger Williams University
Macon State University
Gustavus Adolphus College
University of Alabama at Birmingham
Swarthmore College
University of New Mexico
DePauw University
5
AP United States Government, July 19, 2008
Composite Scores Corresponding to Each
Letter Grade (120-point scale)
Colleges
Exam
Grade
Average
Score
A
B
C
D
F
71
58
45
35
NA*
AP Students
Minimum Composite
AP Grade
Score
5
4
3
2
1
86
69
52
27
0
Average
Score
93
76
60
40
16
*Cases of college students to whom professors assigned an F for either the exam or the course
6 were removed from the analysis
AP United States Government, July 19, 2008
What were the results of the study?
• AP students at the 5-4 cut point scored higher than the
average score earned by a college student at the A level
• AP students at the 4-3 cut point scored higher than the
average score earned by a college student at the B level
• AP students at the 3-2 cut point scored higher than the
average score earned by a college student at the C level
• AP students at the 2-1 cut point scored higher than the
average score earned by a college student at the D level
7
AP United States Government, July 19, 2008
Percentage of Students Earning Each AP Grade
AP Grade
2008 Grade
Distribution
Without the
Comparability
Study*
2008 Actual Grade
Distribution
5
4
3
2
1
7.6%
17.7%
28.1%
30.5%
16.1%
12.2%
13.1%
25.3%
25.8%
23.6%
* 8Percentages based on most but not all of the 2008 data
AP United States Government, July 19, 2008
What are the implications of the study for the
future of AP US Government and Politics?
• The study indicated no serious discrepancy
between AP US Government and Politics
course content and grading standards and
standards used by college courses in the
survey
9
AP United States Government, July 19, 2008
What can AP US Government and Politics
teachers do?
• Keep doing what they’re doing!
• Pay close attention to the course
description and supporting materials
• Work on developing students’ writing and
analytic skills
10
AP United States Government, July 19, 2008