Chapter 4
Alignment Among Secondary and Postsecondary Assessments
in Georgia
The Georgia Assessment Environment
Over the past 10 years, Georgia’s testing program has undergone many changes.
In 1991, the Georgia Assembly established a new set of tests, the Georgia High School
Graduation Tests (GHSGTs), which replaced the previous graduation exam, the
minimum competency Basic Skills Test. The GHSGTs are intended to measure the
learning objectives specified in the state standards (i.e., Quality Core Curriculum), and to
determine whether a student should receive a high school diploma. The new GHSGTs
differed from the previously required Basic Skills Test in that they assessed achievement
in social studies and science (in addition to reading, mathematics, and writing). State law
also mandated that the GHSGTs be a more rigorous test than the Basic Skills Test, and
extended the range of item difficulty found on the GHSGTs.
The GHSGTs graduation tests were phased in gradually. The first group of
students to be affected was those who entered the ninth grade between July 1991 and July
1993 (these would have been students expected to graduate in spring 1995 and 1996).
These students had to pass the English/language arts, mathematics, and writing tests to be
eligible to receive a diploma. The next group, students who entered the ninth grade
between July 1993 and July 1994 (the expected class of '97), was required to pass the
social studies test. And finally, the science test was required for those students who
entered the ninth grade after July 1994 (expected class of '98).
Students have five opportunities to take each of the tests before the end of the
12th grade. There are two testing periods during the junior year. The Writing test is
offered in the early fall semester and again in the spring semester for those students who
need to retake it. Content-specific exams are administered only during the spring
semester of the junior year. After that, the students have three more opportunities to pass
the writing test (summer, fall and spring of their senior year) and the content-specific
exams (summer, fall and spring of their senior year). Students who do not pass all the
required tests but have met all other graduation requirements are eligible to receive a
Certificate of Performance or a Special Education Diploma. These students may then
63
return to take any graduation test(s) necessary in order to qualify for a high school
diploma. There is no limit for these students on the number of times they may retake any
of the tests.
Currently, there are plans to replace the GHSGTs with new assessments
developed as part of the End-of-Course Testing (EOCT) program. Slated for full
implementation in 2003, the EOCT will be multiple-choice, end-of-course exams
assessing student knowledge of course standards in eight core courses, specifically math
(algebra and geometry), ELA (ninth grade language and composition and American
language and composition), social studies (U.S. history and economics/business), and
science (biology and physical science). Although there are plans to make passing some
or all of the end-of-course tests a condition for graduation, there has not been a final
decision regarding how the ECOT will affect graduation requirements.
Georgia Assessments Included in this Study
For this study, we did not include any EOCT assessments because they were not
available when this study was initiated. However, we were able to examine the reading,
editing, math, and writing sections of the GHSGTs. The reading, editing, and math
sections of the GHSGTs assess student knowledge with multiple-choice items, whereas
the writing section requires a written composition. Although each section of the test has
a recommended time limit, the time limit may be extended at the discretion of the test
proctor if students appear to be making progress and need additional time.
In addition to the GHSGTs, students applying to a college in Georgia are often
required to take placement tests in math and/or English. Because the kinds of placement
tests given are likely to vary by the selectivity of the institution, we obtained placement
tests from both a highly selective university (e.g., University of Georgia) and a less
selective institution (e.g. community colleges). Many of the community colleges in
Georgia administer the COMPASS, which is a system of computer-adaptive assessments
that measure student readiness for a broad range of math and English courses.1 Unlike
traditional paper-and-pencil tests, which administer the same set of items to all
examinees, computer-adaptive measures administer different items to examinees based
1
We examined a set of secured items from the COMPASS item pool. COMPASS is published by ACT.
64
on their demonstrated proficiency level. Because students at different proficiency levels
may encounter differ item content mixes, findings about content coverage or cognitive
demands should be interpreted with caution.
For ELA, we examined assessments in writing skills and reading. For math, we
examined tests in numerical skills/prealgebra and algebra. The COMPASS Writing
Skills, COMPASS Reading, and COMPASS Numerical Skills/Prealgebra tests are used
to determine whether students need to enroll in remedial courses, but the COMPASS
Algebra can be used more broadly to place students into an appropriate math course up to
trigonometry.
The University of Georgia does not use a commercial placement test, but instead
administers their own departmentally-created post-admittance exams; we include their
math and English placement tests. At the University of Georgia, all incoming freshmen,
except those with AP Calculus credit or those who have transferred college-level calculus
credit from another postsecondary institution, are required to take a 45-minutes, 26-item
multiple-choice exam, which is used to place students in an appropriate mathematics
course (hereafter this test will be referred to as the UOG Math Placement Test). The
analogous English test, referred to hereafter as the UOG ELA Placement Test, consists of
two sessions: a morning session in which students are given 55 minutes to complete 60
multiple-choice items, and an afternoon session in which students are to compose an a
writing sample in one hour. The writing session is required only of students whose
performance during the morning session is below a predetermined cut-off score.
High school students may also choose to take the Georgia Early Math Placement
Test (GEMPT) in their junior year. The purpose of this measure is to assess students’
current mathematics proficiency level in relation to the skills required by college-level
mathematics courses. Students are made aware of deficiencies that would require them to
take a remedial mathematics course in college, and are encouraged to take the appropriate
preparatory courses during their senior year in high school so that the students do not
have to take the remedial course at the postsecondary level.
Tables 4.1 and 4.2, organized by test function, list these testing programs and the
type of information we were able to obtain for this study. For most tests, we used a
single form from a recent administration or a full-length, published sample test. In a few
65
instances where full-length forms were unavailable, we used published sets of sample
items. This was the case for the GHSGTs. For the ELA tests, Table 4.2 specifies
whether the test includes each of three possible skills: reading, editing, and writing
66
67
College
admissions
College
admissions
SAT II
Mathematics Level
IC
SAT II
Mathematics Level
IIC
College
admissions
ACT
College
admissions
State achievement
Georgia High
School Graduation
Test (GHSGT)
SAT I
Test Type
Test
Full sample
form
Full sample
form
Full sample
form
Full sample
form
Sample items
Materials
Examined
60 minutes
60 minutes
75 minutes
60 minutes
180
minutes
Time Limit
67
Table continues
50 MC
50 MC
35 MC
15 QC
10 GR
60 MC
60 MC
Number and
Type of Items
Calculator
Calculator
Calculator
Calculator
Calculator
Tools
Selection of
students for higher
education
Selection of
students for higher
education
Selection of
students for higher
education
Selection of
students for higher
education
Monitor student
achievement
toward stateapproved content
standards
Purpose
Table 4.1
Technical Characteristics of the Mathematics Assessments
Algebra (18%), geometry (20%,
specifically coordinate (12%)
and three-dimensional (8%)),
trigonometry (20%), functions
(24%), statistics and probability
(6%), and miscellaneous (12%)
Algebra (30%), geometry (38%,
specifically plane Euclidean
(20%), coordinate (12%), and
three-dimensional (6%)),
trigonometry (8%), functions
(12%), statistics and probability
(6%), and miscellaneous (6%)
Arithmetic (13%), algebra
(35%), geometry, (26%), and
other (26%)
Prealgebra (23%), elementary
algebra (17%), intermediate
algebra (15%), coordinate
geometry (15%), plane geometry
(23%) and trigonometry (7%)
Number and Computation (1719%), Data Analysis (19-21%),
Measurement and Geometry (3234%), Algebra (28-30%)
Content as Specified in Test
Specifications
68
College placement
College placement
COMPASS
Numerical
Skills/Prealgebra
Georgia Early
Math Placement
Test (GEMPT)
Notes.
MC = multiple-choice
OE = open-ended
GR = grid-in
QC = quantitative comparison
College placement
College placement
COMPASS
Algebra
University of
Georgia Math
Placement Test
(UOG Math
Placement Test)
Test Type
Test
Full sample
form
Full sample
form
Full sample
form
Full sample
form
Materials
Examined
45 minutes
45 minutes
No time
limit
No time
limit
Time Limit
68
26 MC
32 MC
Number of MC
items given
depends on
ability
Number of MC
items given
depends on
ability
Number and
Type of Items
Calculator
Calculator
None
None
Tools
Placement of
students into
appropriate math
course
Assess current
mathematics
proficiency level in
relation to collegelevel mathematics
courses
Assess proficiency
in prealgebra
Assess student
proficiency in
algebra,
intermediate
algebra, and
coordinate
geometry
Purpose
Arithmetic, algebra, geometry,
trigonometry (see
http://www.math.uga.edu/~curr/
PlacementTopics.html for a
complete list of topics)
Prealgebra, elementary algebra,
geometry, intermediate algebra
Prealgebra
Algebra, intermediate algebra,
coordinate geometry
Content as Specified in Test
Specifications
69
College
admissions
College
admissions
AP Language and
Composition
SAT I
State
achievement
Georgia High School
Writing Test
(GHSWT)
College
admissions
State
achievement
Georgia High School
Graduation Test
(GHSGT)
English/Language Arts
ACT
Test Function
Test
Full sample
form
Full sample
form
Full sample
form
Sample
items
Sample
items
Materials
Examined
75 minutes
180 minutes
--60 minutes
reading
-- 120 minutes
writing
80 minutes
--35 minutes
reading
--45 minutes
editing
90 minutes for
writing section
Additional time
allowed if
student is
making progress
180 minutes for
both reading and
editing
Additional time
allowed if
student is
making progress
Time Limit
69
Table continues
40 MC reading
38 MC editing
52 MC reading
1 OE reading
2 OE writing
40 MC reading
75 MC editing
1 OE writing
50 MC for both
reading and
editing
Number and
Type of Items
Selection of students for
higher education
Provide opportunities for
HS students to receive
college credit and
advanced course
placement
Selection of students for
higher education
Monitor student
achievement toward state
standards, provide high
school diploma
Monitor student
achievement toward state
standards, provide high
school diploma
Purpose
Y
Y
Y
N
Y
Reading
Section?
Table 4.2
Technical Characteristics of the English/Language Arts Assessments
Y
N
Y
N
Y
Editing
Section?
N
Y
N
Y
N
Writing
Section?
70
College
admissions
College
admissions
College
placement
College
placement
College
placement
SAT II Literature
SAT II Writing
COMPASS Reading
COMPASS Writing
Skills
University of Georgia
English/Language Arts
Placement Test (UOG
ELA Placement Test)
Notes.
MC = multiple-choice
OE = open-ended
Test Function
Test
Full sample
form
Full sample
form
Full sample
form
Full sample
form
Full sample
form
Materials
Examined
One 55-minute
MC session for
both reading and
editing
One 60-minute
writing session
No time limit
No time limit
60 minutes
-- 40 minutes
editing
-- 20 minutes
writing
60 minutes
Time Limit
70
60 MC for both
reading and
editing
1 OE writing
Number of MC
questions given
depends upon
ability
Number of MC
questions given
depends upon
ability
60 MC editing
1 OE writing
60 MC reading
Number and
Type of Items
Assess whether admitted
students possess entry
level English skills
Assess whether admitted
students possess entry
level editing skills
Assess whether admitted
students possess entry
level reading skills
Selection of students for
higher education
Selection of students for
higher education
Purpose
Y
N
Y
N
Y
Reading
Section?
Y
Y
N
Y
N
Editing
Section?
Y
N
N
Y
N
Writing
Section?
Alignment Among Georgia Math Assessments
In this section, we describe the results of our alignment exercise for the math
assessments. The results are organized so that alignment among tests with the same
function is presented first, followed by a discussion of alignment among tests with
different functions.
Alignment is described by highlighting similarities and differences with respect to
technical features, content, and cognitive demands. That is, we first present how the
assessments vary along characteristics such as time limit, format, contextualized items,
graphs, diagrams, and formulas. We then document differences with respect to content
areas, and conclude with a discussion of discrepancies in terms of cognitive requirements.
Table 4.3 presents the alignment results for the math assessments. The numbers
in Table 4.3 represent the percent of items falling into each category. As an example of
how to interpret the table, consider the SAT I results; 58% of its items are multiplechoice, 25% are quantitative comparisons, and 17% are grid-in items. With respect to
contextualization, 25% of the SAT I questions are framed as a real-life word problem.
Graphs are included within the item-stem on 7% of the questions, but graphs are not
included within the response options (0%), and students are not asked to produce any
graphs (0%). Similarly, diagrams are included within the item-stem on 18% of the
questions, but diagrams are absent from the response options (0%), and students are not
required to produce a diagram (0%). With respect to content, the SAT I does not include
trigonometry (0%), and assesses elementary algebra (37%) most frequently. In terms of
cognitive demands, procedural knowledge (53%) is the focus of the test, but conceptual
understanding (32%) and problem solving (15%) are assessed as well. Results for the
other tests are interpreted in an analogous manner.
71
72
MC
QC
100
0
0
0
SAT II
Math Level 100
IC
SAT II
Math Level 100
IIC
0
0
COMPASS
Numerical
Skills/Pre- 100
algebra
100
100
GEMPT
UOG Math
Placement
Test
0
0
0
0
0
0
17
0
0
GR
0
0
0
0
0
0
0
0
0
OE
.
Format
MC = multiple-choice items
QC = quantitative comparison items
GR = fill-in-the-grid items
0
0
COMPASS
100
Algebra
College Placement Tests
25
58
SAT I
0
100
ACT
College Admissions Tests
GHSGT
State Achievement Test
Test
Format
8
6
41
5
12
18
25
22
62
C
Context
0
0
0
16
12
8
7
5
9
S
8
3
0
0
2
0
0
2
0
0
0
0
0
0
0
0
0
0
P
19
6
0
0
2
26
18
13
21
S
0
0
0
0
0
0
0
0
0
RO
0
0
0
0
0
0
0
0
0
P
Diagrams
27
13
0
16
10
12
1
15
11
M
Table
72continues
0
3
0
5
0
0
8
0
6
G
Formulas
Contextualization
C = contextualized items
RO
Graphs
0
3
88
11
2
2
13
17
25
PA
27
31
0
21
22
10
2
5
2
IA
15
9
0
26
12
12
6
15
9
CG
12
13
0
0
14
28
19
25
29
PG
19
0
0
0
18
4
0
8
0
0
0
12
0
6
8
13
3
18
TR SP
0
0
0
5
12
6
11
5
0
MISC
23
0
12
5
26
34
32
40
29
77
100
82
90
54
58
53
53
69
PK
Cognitive
Demands
CU
Diagrams
S
= graph/diagram within item-stem
RO = graph/diagram within response options
P
= graph/diagram needs to be produced
27
44
0
37
14
30
37
22
18
EA
Content
Table 4.3
Alignment Among the Technical, Content, and Cognitive Demands Categories for the Math Assessments
0
0
6
5
20
8
15
7
2
PS
73
Formulas
M = formula needs to be memorized
G = formula is provided
Notes.
Format
MC = multiple-choice items
QC = quantitative comparison items
GR = fill-in-the-grid items
OE = open-ended items
OE = open-ended items
73
Content Areas
PA = prealgebraTable continues
EA = elementary algebra
IA = intermediate algebra
CG = coordinate geometry
PG = plane geometry
TR = trigonometry
SP = statistics and probability
MISC = miscellaneous topics
Contextualization
C = contextualized items
Cognitive Demands
CU = conceptual understanding
PK = procedural knowledge
PS = problem-solving
Diagrams
S
= graph/diagram within item-stem
RO = graph/diagram within response options
P
= graph/diagram needs to be produced
Alignment Among Tests With the Same Function
State Achievement Tests
Only one state achievement test is included in this analysis, the GHSGT Math. It
is a 3-hour, 60-item, multiple-choice test assessing math knowledge up to geometry.
Most of its items assess procedural knowledge (69%), but a moderate proportion assesses
conceptual understanding as well (29%).
College Admissions Tests
We examined four college admissions tests: the ACT, SAT I, SAT II Math Level
IC, and SAT II Math Level IIC. All tests, except the SAT I, have a one-hour time limit.
The SAT I has a 75-minute time limit. All four exams are also predominantly multiplechoice, although the SAT I includes quantitative comparison (25%) as well as grid-in
(17%) items. Contextualized questions are most prevalent on the SAT I (25%) and least
prevalent on the SAT II Math Level IIC (12%). Students are rarely asked to work with
graphs, and questions that contain graphs within the item-stem constitute no more than
12% of items on the college admissions measures. Questions that include diagrams
within the item-stem are more prevalent, comprising 26%, 18%, and 13% of items on the
SAT II Math Level IC, SAT I, and ACT, respectively. However, questions with
diagrams are infrequent on the SAT II Math Level IIC (2%). Formulas are also
uncommon, but there are differences with respect to the extent to which formulas are
necessary. Whereas the ACT, SAT II Math Level IC, and SAT II Math Level IIC include
items in which a memorized formula is needed (15%, 12%, and 10%, respectively), these
items are largely absent from the SAT I (1%).
Although college admissions exams generally sample from the same content
areas, they do not do so to the same extent. Elementary algebra comprises most of the
SAT I items (37%). The SAT II Math Level IC also emphasizes elementary algebra
(30%), but focuses on planar geometry as well (28%). The ACT shows a similar content
emphasis as that of the SAT II Math Level IC; 22% of its items assess elementary algebra
and 25% assess planar geometry. The SAT II Math Level IIC, on the other hand, draws
from more advanced content areas, such as intermediate algebra (22%) and trigonometry
(18%).
74
In terms of cognitive demands, all four tests assess procedural knowledge to a
similar degree. Procedural knowledge items constitute between 54% and 58% of the
items found on college admissions measures. However, there is more variation among
the exams with respect to emphasis on problem solving. The SAT I and SAT II Math
Level IIC place relatively greater emphasis on problem solving (20% and 15%,
respectively) than do the ACT and SAT II Math Level IC (7% and 8%, respectively).
College Placement Tests
There are four college placement tests in this analysis: the COMPASS Algebra,
COMPASS Numerical Skills/Prealgebra, GEMPT, and UOG Math Placement Test. All
four tests consist only of multiple-choice items, but the COMPASS Algebra and
COMPASS Numerical Skills/Prealgebra are computer-adaptive measures whereas the
GEMPT and UOG Math Placement Test are paper-and-pencil measures. The
COMPASS Numerical Skills/Prealgebra contains many items framed in a real-life
context (41%), but contextualized items are largely excluded from the COMPASS
Algebra, GEMPT, and UOG Math Placement Test (5%, 6%, and 8%, respectively).
Questions in which graphs are included within the item-stem are absent on all of the
college placement measures except the COMPASS Algebra (16%). Items that contain
diagrams are excluded from the COMPASS Algebra and COMPASS Numerical
Skills/Prealgebra, but comprise a small proportion of the GEMPT (6%), and a moderate
proportion of the UOG Math Placement Test (19%). With respect to items requiring a
memorized formula, the COMPASS Numerical Skills/Prealgebra excludes such items,
but the other college placement measures include this type of question to some extent
(13%-27%).
With the exception of the COMPASS Numerical Skills/Prealgebra, which is
narrowly focused on a single topic area, namely prealgebra (88%), the other college
placement measures show a broad range of content sampling. The COMPASS Algebra
and the GEMPT assess elementary algebra most frequently, (37% and 44%, respectively)
but also assess intermediate algebra to a moderate extent (21% and 31%, respectively).
Content differences between these two tests are most evident with respect to sampling of
coordinate geometry, which comprises 26% of items on the COMPASS Algebra, but 9%
75
of items on the GEMPT. As with the COMPASS Algebra and the GEMPT, the UOG
Math Placement Test focuses most on elementary algebra and intermediate algebra (27%
of items on each content area). However, the UOG Math Placement test also assesses
trigonometry (19%), a topic that is devoid from all other college placement measures.
Few differences are observed with respect to cognitive requirements. Procedural
knowledge problems constitute the majority of the college placement measures, ranging
from 77% of the UOG Math Placement Test questions to 100% of the GEMPT questions.
Problem-solving items are uncommon (0%-6%), as are conceptual understanding items,
although conceptual understanding is represented on the UOG Math Placement test to a
moderate extent (23%).
Alignment Among Tests with Different Functions
With the exception of the SAT I, none of the math assessments requires students
to generate their own answers. Excluding the COMPASS Numerical Skills/Prealgebra
(41%), questions framed within a realistic context typically represent a small proportion
of college placement tests (5%-8% if the COMPASS Numerical Skills/Prealgebra is
excluded), a small to moderate proportion of college admissions measures (12%-25%),
and is most prevalent on the state achievement test (62%). Questions that contain graphs
within the item-stem are relatively uncommon, comprising no more than 16% on any of
the studied exams. Questions that contain diagrams within the item-stem are more
prevalent, but typically represent only a small or moderate fraction of a test. Questions
that contain diagrams within the item-stem represent 2%-26% of college admissions
items, 0%-19% of college placement items, and 21% of the GHSGT. Items requiring a
memorized formula are generally rare, and comprise less than 16% on all but one test.
The exception is the UOG Math Placement Test, where memorized formulas constitute
27% of its items.
With respect to the content category, college admissions exams assess logic
(coded as miscellaneous) and trigonometry more frequently than do the state achievement
or college placement measures. Trigonometry items, for example, are included on 4%18% of college admissions tests (if the SAT I is excluded), 0% of the GHSGT, and 0% of
college placement tests, the UOG Math Placement Test notwithstanding. The
76
anomalously high proportion of trigonometry items found on the UOG Math Placement
Test (19%) reflects the fact that this exam is used to place students into calculus, whereas
the other college placement measures are used to place students into either remedial
courses (COMPASS Numerical Skills/Prealgebra) or math courses no higher than
trigonometry (COMPASS Algebra). The state achievement test, the GHSGT, is broadly
distributed across a range of content areas, particularly prealgebra and planar geometry
(25% and 29%, respectively).
In terms of cognitive requirements, all tests emphasize procedural knowledge
most frequently, although there is variation with respect to extent. Procedural knowledge
items are most common on college placement tests (77%-100%), followed by the state
achievement test, the GHSGT (69%), and least common on college admissions tests
(53%-58%). Problem-solving items are most prevalent on college admissions (7%-20%)
and are rarely included on either the GHSGT (2%) or college placement tests (0%-6%).
Conceptual understanding items are also most likely on college admission exams,
comprising between 26%-40% of the items.
Discussion
Below, we discuss the implications of the discrepancies among the math
assessments. We begin by highlighting instances in which differences are justified, then
address whether there were any misalignments that may send students confusing signals.
We also explore the possibility that state achievement tests can inform postsecondary
decisions.
Which Discrepancies Reflect Differences in Test Use?
As noted in Chapter 1, content discrepancies may reflect differences in intended
test use. To illustrate, consider the SAT II Math Level IIC and GHSGT. Although both
the SAT II Math Level IIC and GHSGT include topics from a wide variety of courses,
the SAT II Math Level IIC includes many trigonometry and problem-solving items (18%
and 20%, respectively), whereas the GHSGT rarely includes such material (0% and 2%,
respectively). In this particular case, the SAT II Math Level IIC and GHSGT have
disparate functions, and content differences reflect variations in purpose. Because the
77
SAT II Math Level IIC is used to select among higher-achieving students for entrance
into universities and colleges, it includes many problem-solving and trigonometry items
in order to distinguish among higher-proficiency examinees. The GHSGT, on the other
hand, is used to monitor student achievement statewide, and therefore requires items of
more moderate difficulty that can be attempted by students with a wider range of
proficiency levels.
The above example represents justifiable discrepancies across tests with different
purposes. However, there are also instances in which discrepancies within tests of
similar purposes are warranted as well. As mentioned earlier, that trigonometry is
prevalent on the UOG Math Placement Test but absent from the COMPASS Algebra is
justifiable given that the former test is used to place students into calculus, whereas the
latter test is used to place students into math courses no higher than trigonometry.
Likewise, differences between the SAT I and ACT with respect to emphasis on problemsolving or non-routine logic problems is indicative of the fact that the SAT I is intended
to be a reasoning measure, but the ACT is intended to assess content found in high-school
math courses.
Is There Evidence of Misalignment?
As defined in Chapter 1, misalignments refer to those discrepancies that are not
attributable to test function, and therefore send students confusing signals regarding the
kinds of skills that are needed to perform well on a given test. In our analysis of the
math tests, we could not find any examples of misalignments, as discrepancies among
college admissions, college placement, and state achievement measures appear to have
stemmed from variations in test use. Additionally, differences among tests of similar
purposes are either small or moderate, or reflect nuances in purpose (e.g., see the above
UOG Math Placement Test and COMPASS Algebra example).
Can State Achievement Tests Inform Postsecondary Admissions and Course Placement
Decisions?
Although there are many discrepancies among exams of different functions, it
may still be possible that a test can serve multiple purposes satisfactorily. Currently,
78
some measures are used for more than one purpose. Many postsecondary institutions, for
example, allow students to submit scores from college admissions exams such as the SAT
I or ACT as a means of exemption from a remedial college placement test. Potentially,
state achievement tests can be used for similar purposes, but no postsecondary institution
to date has made use of GHSGT scores for placement decisions. However, policymakers
have advocated using scores on some graduation tests for purposes beyond monitoring
student achievement (Olson, 2001b; Schmidt, 2000) because such a policy change would
not only reduce testing burden, but it would also motivate students to focus on state
standards rather than on external tests like the SAT I or ACT (Healy, 2001; Olson, 2001a;
Standards for Success, 2001). Below, we discuss the potential of the GHSGT for guiding
college placement and admissions decisions. We also discuss the possibility that testing
burden can be eased by using scores from the SAT II Math Level IIC for other purposes,
namely placement into an appropriate math course.
It may be possible to use GHSGT scores to exempt students from remedial
courses, as its content is more rigorous than that of the COMPASS Numerical
Skills/Prealgebra. Because the GHSGT assesses elementary algebra and geometry more
frequently than does the COMPASS Numerical Skills/Prealgebra, a sufficiently high
score on the GHSGT should logically excuse students from having to take the
COMPASS Numerical Skills/Prealgebra.
However, the GHSGT cannot be used to place students into an appropriate math
course because it rarely assesses material beyond geometry. Compared to the
COMPASS Algebra (which assesses intermediate algebra) or the UOG Placement Test
(which assess trigonometry), the GHSGT would not provide sufficient information
regarding whether an examinee has the necessary background to enroll in a high-level
math course such as trigonometry or calculus.
For the same reason, the GHSGT is not a viable alternative to any of the college
admissions measures. It contains fewer intermediate algebra and trigonometry items
than either the SAT II Math Level IC or SAT II Math Level IIC, and contains a lower
proportion of problem-solving items than either the SAT I or ACT. Because the GHSGT
does not sample as extensively from these content areas, its discriminating power is likely
to be more limited than that of the college admissions exams.
79
It may be possible to ease students’ testing burden in other ways beyond
expanding the use of the GHSGT scores. For instance, the SAT II Math Level IIC can be
used to place students into an appropriate math course up to calculus. The SAT II Math
Level IIC contains approximately the same proportion of intermediate algebra items as
the COMPASS Algebra, as well as the same proportion of trigonometry items as the
UOG Math Placement Test. The SAT II Math Level IIC also has the added advantage of
assessing problem-solving to a greater extent than do these two other tests. Conceivably,
academic counselors can use the scores from the SAT II Math Level IIC to advise
students which math course is most appropriate. To determine the feasibility of the SAT
II Math Level IIC as a measure that informs placement decisions, more research is
needed to explore the interrelationships between the SAT II Math Level IIC, the UOG
Math Placement Test, and grades in higher-level math courses (i.e., calculus).
Alignment Among Georgia ELA Assessments
Below we present the ELA results. As with math, we discuss discrepancies both
within and across test functions. The results are also organized by skill, namely reading,
editing, and writing. In some instances, there are only two tests that assess the same skill
and share the same function, so it is important to recognize that patterns or comparisons
between these tests may not be indicative of more general trends within this category of
tests.
Alignment is characterized by describing differences with respect to technical
features, content, and cognitive demands. Specifically, we discuss differences in time
limit and format, then document discrepancies with respect to topic, voice, and genre of
the reading passages, before concluding with variations in cognitive processes.
The alignment results for tests that measure reading skills are presented in Tables
4.4-4.5. Tables 4.6-4.7 provide the results for exams that assess editing skills, and Tables
4.8-4.9 provide the findings for exams that assess writing skills. For each table, the
numbers represent the percent of items falling in each category. To provide a concrete
example of how to interpret the findings, consider the content category results for the AP
Language and Composition, presented in Table 4.4. With respect to topic, 50% of the
reading passages included on the AP Language and Composition are personal accounts,
80
whereas 25% of the topics are about humanities, and the remaining 25% are about natural
science. It does not include topics from fiction or social science (0% each). In terms of
the author’s voice, 75% of the passages are written in a narrative style, whereas the other
25% are written in an informative manner. With respect to genre, only essays (100%) are
used; passages on the AP Language and Composition are not presented as letters, poems,
or stories (0% each). Results for the other tests are interpreted in a similar manner.
81
82
0
20
63
AP Language and
Composition
SAT I
SAT II Literature
0
UOG ELA Placement Test
50
25
0
40
25
25
9
0
25
0
20
25
25
9
0
25
13
20
0
25
14
Humanities Natural Social
Science Science
Note.
* One passage contained both an essay and a poem.
13
COMPASS Reading
College Placement Tests
25
45
Fiction
ACT
College Admissions Tests
GHSGT
State Achievement Test
Test
Topic
50
13
25
0
50
0
23
Personal
Accounts
82
50
38
63
20
0
25
73
0
0
0
40
25
25
0
Narrative Descriptive
0
0
0
20
25
25
2
Persuasive
Voice
50
63
13
20
0
25
25
Informative
Table 4.4
Alignment Within the Content Category for the Reading Passages
0
0
13
0
0
0
2
Letter
100
88
25
80
100
75
55*
Essay
0
0
50
0
0
0
16*
Poem
Genre
0
13
13
20
0
25
30
Story
Reading Measures
Alignment Among Tests of the Same Function
State Achievement Tests
The only state achievement test included in our sample is the GHSGT, which
assesses reading proficiency solely with multiple-choice items. Because the GHSGT
does not contain separate sections for editing and reading items, we cannot determine
testing time earmarked specifically for assessing reading skills, although testing time
devoted to assessing both types of skills is 180 minutes (see Table 4.2).
Most of the reading passages on the GHSGT are fiction (45%), written in a
narrative style (73%), and presented as an essay (55%) (see Table 4.4). The GHSGT
reading items assess inference skills (70%) most frequently, although a moderate
proportion of questions also assess recall skills (30%) (see Table 4.5).
Table 4.5
Alignment Within the Cognitive Demands Category for Tests
Measuring Reading Skills
Test
Recall
Inference
Evaluate Style
30
70
0
ACT
58
42
0
AP Language and Composition
23
77
0
SAT I
18
83
0
SAT II Literature
13
80
7
COMPASS Reading
75
25
0
UOG ELA Placement Test
46
54
0
State Achievement Test
GHSGT
College Admissions Tests
College Placement Tests
College Admissions Tests
Four college admissions exams assess reading proficiency: the ACT, AP
Language and Composition, SAT I, and SAT II Literature. With the exception of the AP
Language and Composition, no college admissions test assesses reading skills with openended items. Testing time devoted to measuring reading skills is 60 minutes for both the
83
SAT II Literature and AP Language and Composition. Because the SAT I does not
contain separate sections for editing and reading items, we cannot determine testing time
earmarked specifically for assessing reading proficiency, although testing time devoted to
assessing both types of skills is 75 minutes (see Table 4.2).
Reading passage topics also vary from one measure to the next (see Table 4.4).
The SAT II Literature emphasizes fiction (63%) whereas the AP Language and
Composition emphasizes personal accounts (50%). The SAT I favors humanities (40%),
but the ACT is evenly distributed among fiction, humanities, natural science, and social
science (25% each). Narrative pieces are included on all college admissions measures,
and range from 40% of the SAT I passages to 100% of the SAT II Literature passages.
Essay is generally the most common genre, appearing on 75% of the ACT, 80% of the
SAT I, and 100% of the AP Language and Composition passages. However, the SAT II
Literature is more likely to include poems (50%) than essays (25%). With the exception
of the ACT, college admission exams place greatest emphasis on interpretation and
analysis of the reading passages. Inference items range from 42% of the ACT questions
to 83% of the SAT I questions (see Table 4.5).
College Placement Tests
Two college placement tests, the COMPASS Reading and the UOG ELA
Placement Test, contain reading items. Both assess reading proficiency with the
multiple-choice format. As with the GHSGT and SAT I, the UOG ELA Placement Test
does not specify testing time devoted specifically to measuring reading proficiency.
Because the COMPASS Reading is a computer-adaptive measure, testing time varies by
examinee (see Table 4.2).
Reading topics on the COMPASS Reading are distributed among humanities,
natural science, and social science (25% each), but reading topics on the UOG ELA
Placement Test are distributed between humanities and personal accounts (50% each)
(see Table 4.4). With respect to author’s voice, the COMPASS Reading favors
informative works (63%), whereas the UOG ELA Placement Test is evenly split between
narrative and informative pieces (50% each). Essay is the most commonly used genre on
either test (88%-100%). For the cognitive demands category, the COMPASS Reading
84
emphasizes recall to a great extent (75%), but the UOG ELA Placement Test focuses on
both recall and inference (46% and 54%, respectively) (see Table 4.5).
Alignment Among Tests of Different Functions
Across all measures, reading skills are assessed primarily with multiple-choice
items. Among the assessments that contain separate sections for reading, testing time
ranges from 35 minutes for the ACT to 60 minutes for the SAT II Literature and AP
Language and Composition. All assessments contain reading passages on two or more
topics, and every reading assessment includes a topic from humanities except the SAT II
Literature. Every measure also includes both narrative and informative passages. Essay
is the most prevalent genre, comprising 55% of the GHSGT, 88% -100% of college
placement, and 75%-100% of college admissions tests, the SAT II Literature exam
notwithstanding. Instead, the SAT II Literature favors poems. Both the GHSGT (70%)
and most of the college admissions measures focus heavily on inference questions (77%83%, excluding the ACT), whereas college placement measures place relatively greater
emphasis on recall (46%-75%).
Editing Measures
Alignment Among Tests of the Same Function
State Achievement Test
The GHSGT is the only measure that falls within this category. Again, we cannot
determine testing time devoted to assessing editing skills because editing and reading
items are combined within a single section (see Table 4.2). As with reading, editing
proficiency is assessed only with multiple-choice items. Most of the passages included
on the GHSGT editing section are fictional works (60%) in which the author uses a
narrative voice (80%). Story is the most common genre (60%), although a significant
number of passages are also presented as essays (40%) (see Table 4.6). With respect to
cognitive demands, inference skills are assessed most frequently (44%), although a
moderate proportion of items also assess recall (30%) and evaluate style (26%) skills.
85
86
UOG ELA Placement Test
COMPASS Writing Skills
College Placement Tests
SAT II Writing
SAT I
ACT
College Admissions Tests
GHSGT
State Achievement Test
Test
0
0
0
60
Fiction
0
100
60
20
N/A
50
0
N/A
20
20
50
0
0
0
Humanities Natural Social
Science Science
Topic
0
0
20
0
Personal
Accounts
86
0
50
40
80
0
0
0
0
N/A
N/A
0
0
0
0
Narrative Descriptive Persuasive
Voice
100
50
60
20
Informative
Table 4.6
Alignment Within the Content Category for the Editing Passages
0
0
0
0
Letter
100
100
100
40
N/A
0
0
0
0
Poem
N/A
Essay
Genre
0
0
0
60
Story
Table 4.7
Alignment Within the Cognitive Demands Category for Tests
Measuring Editing Skills
Test
Recall
Inference
Evaluate Style
30
44
26
ACT
48
4
48
SAT I
0
100
0
SAT II Writing
50
3
47
COMPASS Writing Skills
80
3
17
UOG ELA Placement Test
100
0
0
State Achievement Test
GHSGT
College Admissions Tests
College Placement Tests
College Admissions Tests
Items measuring editing skills are included on three college admissions tests, the
ACT, SAT I, and SAT II Writing. The exams are predominantly multiple-choice, with
testing time ranging from 40 minutes for the ACT to 45 minutes for the SAT II Writing
(see Table 4.2). As mentioned earlier, the SAT I does not specify the specific amount of
testing time devoted to measuring editing skills.
The SAT I does not include a reading passage, but instead uses a few sentences as
item prompts. In contrast, the ACT and SAT II Writing include reading passages. These
reading passages are typically essays about humanities, and written in either a narrative
or informative voice (see Table 4.6). The ACT and SAT II Writing items are equally
distributed among recall (48% and 50%, respectively) and evaluate style items (48% and
47%, respectively), but the SAT I assesses only inference skills (100%) (see Table 4.7).
College Placement Tests
Two college placement tests, the COMPASS Writing Skills and UOG ELA
Placement Test, assess editing skills via multiple-choice items. Neither test specifies the
time limit devoted solely to measuring editing skills (see Table 4.2). The UOG ELA
Placement Test uses sentences to assess editing skills, but the COMPASS Writing Skills
includes passages. Topics on the COMPASS Writing Skills are evenly split between
87
natural science and social science (50%), and passages are typically essays (100%) in
which the author uses an informative voice (100%) (see Table 4.6). In terms of cognitive
demands, both tests emphasize recall items (80%-100%).
Alignment Among Tests of Different Functions
Across all measures, editing skills are assessed solely with multiple-choice items.
All tests, except the SAT I and UOG ELA Placement Test, include reading passages.
(The SAT I and UOG ELA Placement Test use sentences as prompts). Of those
measures that include a reading passage, all include an essay, written in a narrative or
informative voice. More variation is observed with respect to reading passage topics.
The GHSGT favors fiction (60%) whereas humanities topics are most likely to appear on
the ACT or SAT II Writing (60% and 100%, respectively). In contrast, natural science
and social science are the most frequent topics on the COMPASS Writing Skills (50%
each topic).
Both the college placement and college admissions exams tend not to span the full
spectrum of the cognitive demands category. The ACT and SAT II Writing emphasize
recall and evaluate style items, but are generally devoid of inference items, whereas the
reverse is true for the SAT I. College placement measures focus heavily on recall items
(80%-100%) and rarely include inference and evaluate style items. In contrast, the
GHSGT assesses all three levels of the cognitive demands category; the test is almost
evenly distributed among recall, inference, and evaluate style items (30%, 44%, and 26%,
respectively).
Writing Measures
Alignment Among Tests of the Same Function
State Achievement Tests
The GHSWT requires students to provide one writing sample within a 90-minute
testing session (see Table 4.2). Topics are typically drawn from humanities or personal
accounts (see Table 4.8). With respect to scoring criteria, GHSWT requires students to
demonstrate mechanics, word choice, style, organization, and insight (see Table 4.9).
88
Table 4.8
Alignment Among the Writing Prompt Topics
Topic
Test
Fiction
Humanities
Natural
Science
Social
Science
Personal
Accounts
State Achievement Test
GHSWT
X
X
AP Language and Composition
X
X
SAT II Writing
X
College Admissions Tests
College Placement Test
UOG ELA Placement Test
X
X
X
X
Table 4.9
Alignment Among the Scoring Criteria for Tests Measuring Writing Skills
Scoring Criteria Elements
Test
Mechanics
Word Choice
Organization
Style
Insight
X
X
X
X
X
AP Language and Composition
X
X
X
X
X
SAT II Writing
X
X
X
X
X
X
X
X
State Achievement Test
GHSWT
College Admissions Tests
College Placement Test
UOG ELA Placement Test
X
College Admissions Tests
Of the college admissions measures, only the SAT II Writing and AP Language
and Composition require a writing sample. The SAT II Writing provides students with a
one- or two-sentence writing prompt on a topic (usually humanities), and allows 20
minutes for students to respond (see Tables 4.2 and 4.8). In contrast, prompts on the AP
Language and Composition are typically reading passages, and students are required to
provide a total of three writing samples in over two hours (see Table 4.2).2 Topics can
2
The AP Language and Composition requires a total of three writing samples, two of which are produced
during the 120-minute writing session, and one during the 60-minute reading session. However, because
examinees also respond to a set of multiple-choice items during the reading session, it is unknown the
amount of time students devote specifically to the writing sample.
89
vary, but are usually about humanities or personal accounts (see Table 4.8). The AP
Language and Composition emphasizes all elements of the scoring criteria, but the SAT
II Writing downplays the importance of insight (see Table 4.9).
College Placement Tests
Only one college placement exam, the UOG ELA Placement Test, requires a
written composition. Students are to write a single composition within a 60-minute
session (see Table 4.2). Topics vary widely from humanities and natural science to social
science and personal accounts (see Table 4.8). The scoring rubric emphasizes mechanics,
word choice, organization, style, and insight.
Alignment Among Tests of Different Functions
Writing measures can vary from 20 minutes for a single writing sample (SAT II
Writing) to over 2 hours for three writing samples (AP Language and Composition).
Humanities and personal accounts are the most common topic, as every test includes a
writing prompt from one, if not both, of these areas. All but one writing test emphasizes
mechanics, word choice, organization, style, and insight. The exception is the SAT II
Writing, which omits insight from its scoring criteria.
Discussion
Our discussion of the discrepancies among ELA assessments parallels that of the
math discussion. We first identify examples of discrepancies that are justifiable, then
discuss the implications of the misalignments. We also discuss the feasibility of using
state achievement tests to inform admissions decisions.
Which Discrepancies Reflect Differences in Test Use?
As in math, some discrepancies among the ELA assessments reflect differences in
purpose. Consider, for instance, discrepancies between the scoring standards of the
GHSWT and the AP Language and Composition. For the former test, maximum scores
are awarded to writing samples that have minor diction errors, mechanics lapses, and
underdeveloped paragraphs. Under the AP Language and Composition guidelines, such
90
compositions might receive adequate scores, but would not be viewed as exemplary
papers. Because the AP Language and Composition is used to award academic credit to
students who demonstrate college-level proficiency, whereas the GHSWT is used to
monitor the achievement of all students within the state, including those not planning to
attend a postsecondary institution, discrepancies between their scoring criteria are
warranted.
Even when two measures have similar test functions, discrepancies may still be
warranted. For example, the large discrepancy between the SAT I (100%) and the ACT
(4%) and the SAT I and SAT II Writing (3%) with respect to inference items is
attributable to subtleties in purpose. The SAT I is intended to be a measure of reasoning
proficiency, so great emphasis on inference questions is justifiable. The ACT and SAT II
Writing, on the other hand, are curriculum-based measures, so relatively greater focus on
skills learned within English classes (i.e., recall and evaluate style skills) is to be
expected.
Is There Evidence of Misalignment?
Although the majority of the ELA discrepancies is small or moderate, or stems
from variations in test function, one instance of misalignment pertains to the scoring
criteria of the SAT II Writing. Insight is included within the scoring criteria of the
GHSWT, AP Language and Composition, and UOG ELA Placement test, but is omitted
from the scoring rubrics of the SAT II Writing. Given that insight is included in the
standards of most English courses, it appears that the SAT II Writing standards are
incongruent with those that are typically expressed. Potentially, this misalignment can
send students mixed messages about the importance of insight with respect to writing
skills. If the developers of the SAT II Writing were to add insight to the scoring criteria,
or provided a clear rationale of why insight has been omitted from the scoring rubrics,
students would receive a more consistent signal about the importance of insight with
respect to writing proficiency.
91
Can State Achievement Tests Inform Postsecondary Admissions Decisions?
As mentioned earlier, policymakers are exploring the possibility that scores on
graduation tests can be used to inform college admissions decisions. In reading, that the
GHSGT assesses inference skills to approximately the same extent as college admission
measures may suggest that the GHSGT might be a viable alternative to college
admissions tests. However, inference items can vary with respect to cognitive
sophistication elicited. A previous study by Education Trust (1999) showed that ELA
inference items could vary greatly with respect to nuance of interpretations. Given the
differences in the intended test uses, it is very likely that inference items on the GHSGT
may not be as complex as that elicited by college admissions exams. More research
needs to be conducted to determine whether the GHSGT can discriminate among higherachieving examinees as well as college admissions exams.
With respect to assessing writing skills, the GHSWT holds more promise as an
alternative to college admissions tests. Neither the ACT nor the SAT I requires a writing
sample, and the SAT II Writing allows 20 minutes for a writing sample. Given the short
time limit, the SAT II Writing composition represents a very limited indicator of writing
proficiency. The GHSWT, on the other hand, allows students 90 minutes for a
composition. Arguably, the GHSWT would allow admissions officers to better judge
applicants’ writing skills than the ACT, SAT I, or SAT II Writing. However, as
discussed earlier, the current scoring rubrics for the GHSWT may not be rigorous enough
to be of use for some institutions, especially the selective ones. Therefore, changes to the
scoring guidelines may need to be implemented if the GHSWT writing samples were to
be used to inform admissions decisions at these higher-selectivity schools. Again, any
policy changes regarding the use of the GHSWT to inform admissions decisions will
require more research, particularly the relationship between the GHSWT scores and firstyear college grade point average.
92