SYSTEM EFFICIENCY & EFFECTIVENESS
Summary Report:
Statewide Standardized Placement Assessment and Cut Scores
(Instruction Commission)
Work Group members:
Name
Council
Affiliation
Steve Ashpole
Position
College
Registrar
Tacoma CC
Dorna Bullpitt
IC
VP Instruction
SPSCC
David Chalif
ATC
Dean, Math
Edmonds
Rhonda Coats
SSC
Vice President for Student Services
South Puget Sound
Math Coordinator
Renton Technical College
Marty Cooksey
Leslie Eglin
CBS
Basic Skills
Yakima Valley CC
Jamie Fouty
WARP
Institutional Research
Walla Walla
Assessment Center Manager
Spokane Community College
Dean of Student Success
Bellevue College
Institutional Research
Lower Columbia
Kathleen Hathaway
Faculty
Clover Park TC
Heather Keast
Developmental Writing
Spokane Falls CC
Tom McCollow
Math faculty
Pierce, Puyallup
ATC
Dean, General & Developmental
Education
Renton Technical College
Bob Mohrbacher
SSC
VP, Instruction & Student Services
Big Bend
Joe Montgomery
WARP
Dean for Institutional Effectiveness
Columbia Basin
Bill Moore
Policy Assoc, Assessmt, Tchg & Lrng
SBCTC
Regina Reed
Testing Director
Walla Walla
Pam Reising
Math
Green River
Rebecca Goss
Matt Groshong
Wendy Hall
Peggy Moe
WARP
Sam Salvatori
Counselor
Spokane IEL
Lora Senf
CBS
Acting Assistant Dean for ABE
Spokane IEL
Geri Swope
CBS
Dean of Instruction, ABE Division
Spokane IEL
Coordinating Committee Instruction
Tacoma CC
Workgroup
Mike Towey
Jeff Wagnitz
IC
VP Instruction
Highline CC
Kim Ward
CBS
Associate Dean, ABE/ESL
Tacoma CC
1
I.
Current placement assessment practice in Washington CTCs
No comprehensive and detailed assessment of placement assessment practices and policies in
Washington community and technical colleges currently exists. In spring 2011, the Diagnostics,
Assessment and Placement Work Group gathered and compiled some information related to
campus policies in the area (see Appendix A for the results). While there are college variations
in specific policies related to placement, all Washington community and technical colleges use
some kind of standardized test to assess students on entry in math, writing and reading and
then place them in math and English courses based on those scores (reading scores are less
consistently used than math and writing). The specific tests used are determined by individual
colleges; roughly 2/3 of the colleges use COMPASS, an assessment produced by ACT, while the
other 1/3 use ACCUPLACER (from College Board). In the area of math two colleges have
developed their own local placement tests; a few colleges offer multiple test options.
While there are numerous differences in their specific details (e.g., ACCUPLACER is web-based),
both COMPASS and ACCUPLACER are suites of generally computer adaptive tests—the items
presented to students branch and vary based on the number and difficulty of previous correct
response—that can be customized in a variety of ways based on local decisions. This approach
provides for the potential of considerable flexibility as well as efficiency (length of testing
sessions, immediacy of scores) but also can make implementation more challenging. Both test
suites offer diagnostic tests as well as their placement tests but because these tests are more
expensive and more time-consuming colleges rarely include them in their standard
implementation.
Local colleges determine the associated cut scores for student placement into classes based on
their curriculum in math and English; as a result these cut scores vary across the system,
especially in math. Based on the results of a 2008 system survey regarding math placement
practices, most colleges provide some sort of individual advising and score interpretation
immediately after the placement test; allow at least one retest; and offer some kind of
challenge process, varying from talking to the testing manager to formal requests submitted to
a dean or academic department.
II.
Summary of findings related to literature, research, and best practices regarding
higher education placement assessment
Historically entry placement testing served to sort students into classes more or less by skill
level, a process seen as essential for open-admissions institutions like community and technical
colleges. Particularly for community and technical colleges, placement tests reflect and enforce
“hidden” college readiness standards that have a significant impact on students’ lives in terms
2
of costs and opportunities for achieving their career and life goals. With a few exceptions, this
process has been relatively obscure and little-known outside of college testing centers; in the
past decade or so, several factors have converged to focus a much sharper spotlight on the
area, including:
1. Explicit emphasis on educational standards, especially with regard to college- and
career-readiness
2. Policy attention to the lack of articulation and connections across the educational
system
3. Growing understanding of the community and technical college role in providing
educational opportunity to a growing population of students historically under-served
by higher education
4. Recognition of amount and cost of remediation at the postsecondary level, and in
particular the significant “gatekeeper” role played by mathematics
These new perspectives have led to considerable recent research on the quality and
effectiveness of current practice, leading to a number of significant critiques and
recommendations for improvements. A few recent findings/conclusions:







The most commonly-used placement tests (COMPASS and ACCUPLACER) serve as
positive but relatively weak predictors of college-level success; they are much less
effective at identifying who is likely to benefit from what kind of academic
interventions.
A single defined cutoff point on a single tests exaggerates the distinction between
“developmental” and “college-ready.”
Current assessment and placement policies around the country do not consistently
result in better outcomes for students. (Hughes and Scott-Clayton, 2010)
Many two-year college students are confused about and not well-advised during the
placement process. (Nodine, Bracco, & Venezia, 2010)
Students generally take placement tests without full understanding of purpose and
significance.
Placement tests offer little or no information relevant to faculty or classroom
instruction.
Colleges rely solely on single standardized test and have little systematic or ongoing
deliberation about placement choices and issues. (Safran & Visher, 2010)
The thorough review of developmental education placement assessment policies and practices
around the country by Hughes and Scott-Clayton concluded that centralized placement policies
tend to minimize institution-specific factors and have a number of negative unintended
consequences, including many incorrect placements and increased costs due to excessive
remediation. While research is ongoing, the evidence continues to mount that relying on a
3
single standardized test and investing resources in building the perfect cut scores are not
particularly useful solutions to improving the placement assessment process in community and
technical colleges. At the same time, policy and budget pressures around the nature and role of
precollege programs continue to mount as well. In response to those pressures, two-year
colleges and systems around the country are beginning to explore and experiment with a wide
variety of approaches to assessment and placement. Based on this work, a solid consensus is
building around several emerging “good practice” principles:
a) Offering multiple measures, including options like transcripts and “directed selfplacement,” to provide non-test alternatives for entering students;
b) Incorporating a consistent diagnostic focus on strengths and weaknesses rather than
solely on course placement decisions;
c) Providing students with a “college readiness” profile based on multiple indices,
including both academic and affective dimensions;
d) Accelerating student progress into college-level work by enabling students to place into
the highest possible level of precollege studies and still be successful;
e) Collaborating with K-12 partners to offer early assessment, align curricula, and reduce
amount of remediation required by recent high school graduates; and
f) Maintaining faculty involvement in college-level placement processes and decisions.
Selected References
Boylan, H. R. (2009). Targeted Intervention for Developmental Education Students
(T.I.D.E.S). Journal of Developmental Education, 32(3), 14–23.
Collins, M. L. (2008). It's not about the cut score: Redesigning placement assessment
policy to improve student success. Boston, MA: Jobs for the Future.
Hughes, K. L. & Scott-Clayton, J. (2010). Assessing developmental assessment in
community colleges: A review of the literature. (CCRC Working paper No. 19). New York, NY:
Community College Research Center, Teachers College.
Jenkins, D., & Zeidenberg, M.(2007). Developmental education placement policies and
student success in the Connecticut community colleges. New York, NY: Community College
Research Center, Teachers College, Columbia University.
Nodine, T., Bracco, K. R., & Venezia, A. (2010). One shot deal: Students’ perceptions of
assessment and course placement at the California community colleges. San Francisco, CA:
WestEd.
Prince, H. (2005). Standardization vs. flexibility: State policy options on placement
testing for developmental education in community colleges (Policy Brief). Boston, MA: Jobs for
the Future.
Safran, S., & Visher, M. G. (2010). Case studies of three community colleges: The policy
and practice of assessing and placing students in developmental education courses (Working
Paper). New York, NY: National Center for Postsecondary Research and MDRC.
Saxon, D. P., Levine-Brown, P., & Boylan, H. R. (2008). Affective assessment for
developmental students, part I. Research in Developmental Education, 22(1), 1–4.
4
III.
Recommendations for placement and diagnostic assessment
A number of states have implemented or are considering a common process or a single test for
their whole college system, sometimes with common or recommended cut scores, especially
for determing “college readiness.” However, the evidence as to whether system
standardization can reduce costs and improve the effectiveness of placement is minimal and
mixed at best. As Prince (2005) argues, to have the best chance of producing a positive impact
for students such approaches need to be thoughtfully and carefully integrated into a holistic
overhaul of the entire precollege structure and operation. Based on this existing research and
the evidence for the good practice principles noted above, the work group recommends against
standardizing on a single placement test and common cut scores system-wide. Such an
approach would produce a surface and cosmetic solution to a deeper and more complex
problem.
Because the current research literature cited earlier argues strongly in favor of the use of
multiple measures for placement, the overall recommendation is that at each college, students
should have the opportunity to participate in a dynamic and informed assessment process
incorporating multiple, valid predictors of performance. This process will provide both the
institution and the student with the information needed to make informed decisions about
course placement in order for students to move through precollege to college-level work as
quickly as is reasonable. Based on this evidence-based vision of a holistic placement assessment
process, the following specific recommendations will improve system efficiency by reducing the
amount of time students spend in precollege courses and accelerating their progress to and
through college-level coursework while maintaining academic standards and institutional
flexibility.
Recommendation 1: Provide access to multiple measures of readiness in determining student
placement
The community and technical college system should move away from a single standardized test
as the determiner of student placement and instead provide multiple avenues to assess each
student’s preparedness for college work. Rather than relying solely on a single narrow
measure, the goal is to provide opportunities for relying on multiple sources of evidence that
would place the student as high as possible and still promote student success.
To meet this goal, colleges would make available a menu of assessment tools. The available
options may include:





COMPASS, AccuPlacer, or other commercial placement tests, including the diagnostic
components of these tests
Other standardized tests, such as SAT, ACT, or MPT
Affective measures
Locally developed, authentic assessments (e.g., writing samples)
High school transcripts or self-report of prior school performance
5


Directed self-placement or delayed placement options
Credit for prior learning
In particular colleges should be encouraged to incorporate diagnostic assessments into the
placement process. Diagnosis of specific skills is necessary in order to support and promote
innovations like delayed placement and curricular modules that target specific academic
deficiencies rather than requiring all students to take full courses regardless of their specific
skills and needs.
Students would not be required to complete multiple assessments, nor would colleges be
obligated to assess every incoming student in multiple ways; the recommendation is that
placement test scores should not be the only measure available. Colleges should use multiple
assessment methods, as needed, to optimize placement accuracy, and in particular, to minimize
inappropriate placement at developmental levels. Students and college staff should be
encouraged to review placement evidence available for individual students and provide an
opportunity for student input into the placement decision.
Rationale
This recommendation offers several advantages to the system; providing access to multiple
measures or sources of evidence for placement:
 Can be implemented immediately, without requiring colleges to abandon their current
commercial instruments
 Encourages students to participate actively in their own assessment process, increasing
their engagement in that process and commitment to the courses they select
 Promotes college experimentation with innovative placement assessment alternatives such
as directed self-placement, delayed placement, and modularized remediation
 Emphasizes a “highest-best” model that explicitly focuses on helping the student identify
the highest-possible course placement as a starting point
Recommendation 2: Use high school transcripts for English and Math placement
High school students entering Washington community and technical colleges, within a certain
defined time frame after graduation, should be allowed to use their high school transcripts as a
course placement alternative into English and math, based on which courses were completed
successfully and when they were taken.
Rationale
The underlying reason for transforming precollege education in Washington community and
technical colleges is to accelerate students’ progress through precollege coursework and into
college-level courses. For the significant numbers of students who enter the two-year college
system directly from high school (or within 1-2 years), the best way to accelerate this progress
is to help them avoid precollege courses entirely by being better-prepared for college while in
6
high school. Currently a handful of Washington community and technical colleges successfully
use high school transcripts for placement into math classes. This practice should be extended
state-wide (and broadened to include English as well as math) as we implement multiple
measures of placement. Colleges should recognize coursework that students have done in high
school in order to improve college preparation by encouraging rigorous course-taking prior to
college and to build better partnerships with school districts and high school teachers in the
process. The goal is to use the transcript-based process to inform students about what they
can do to achieve readiness in English and mathematics before leaving high school and help
motivate them to take needed steps in high school to achieve college readiness.
Recommendation 3: Adopt a statewide assessment tool that identifies affective skills and
abilities of entering students and provides a portable "college readiness" profile
The system should develop a statewide web-based assessment which addresses both core
academic skills and affective domains (sometimes referred to as “soft skills,” e.g., personal
responsibility, self-motivation and -awareness, time management, teamwork, and emotional
intelligence) that will provide a reliable and valid “college readiness” profile for students
entering the Washington community and technical college system.
Rationale
A. Provide entering students with a clearer and more consistent understanding of their
readiness to do college-level work.
B. Provide colleges with a more comprehensive profile for entering students that can be
incorporated into the course placement process and/or be used to target the level and
nature of intervention needed to help students succeed in college-level work.
Recommendation 4: Create placement reciprocity process across the system
At the same time that the other recommendations are in process, we recommend that the
system should establish a statewide process of reciprocity for college-to-college acceptance of
student placement results. Further, the system should promote and encourage the creation of
local consortia to develop more specific regional placement agreements.
Rationale
Given the well-documented limitations of commercial placement instruments, it is not
surprising that states with universal testing and cut-off scores have not seen appreciable
improvements in student attainment. At the same time, it is also clear that students, high
schools, college staff, and community partners can benefit from greater predictability in how
student placement results are treated from one campus to another. A reciprocity-based system
would provide students with that predictability, while avoiding some of the pitfalls inherent in
commercial placement testing. The reciprocity approach:
7
 Allows for the incorporation of multiple indices — including high school transcript
evaluations and developmental course completion (e.g., “completion of English 09X”) — as
reciprocal indicators of next-course placement;
 Supports a variety of approaches to placement itself, thus encouraging experimentation
with innovative assessment models;
 Leverages the efficiencies of the state’s well-established processes for negotiating and
implementing reciprocity agreements (e.g., transfer distribution requirements and diversity
requirements);
 Reinforces the principle of professional trust that underlies the mutual acceptance of course
work, credentials, and competencies in lateral transfer;
 Supports and leverages system-wide efforts to define “college-readiness” in competencybased and curriculum-based, rather than score-based, terms; and
 Avoids the inefficiencies of cost, distraction, and delays that almost certainly would result
from any statewide effort to identify a common placement instrument and set of cut scores.
Recommendation 5: Provide a comprehensive pre-test process for students, including a)
orientation to purpose and nature of tests and b) skills-based test preparation
Colleges should provide a comprehensive (potentially mandatory, at least for some students,
perhaps based on the results of their “college readiness profile”) orientation preceding the
placement testing process that raises awareness about the importance of the placement
process, communicates to students the high stakes nature of assessment, and provides them
with options and resources for test preparation. Colleges may choose a combination of
methods for implementation that work best for their student population. To facilitate
implementation, colleges will be provided with two key tools: a state of the art video that can
be customized at each campus, explaining to students the purposes of placement testing in the
Washington community and technical college system as well as a checklist of key points to be
covered in an assessment orientation.
Rationale:
Research indicates that many students are not adequately prepared for their placement tests
and/or are often not aware of the ramifications of their assessment results. This lack of
preparation often leads to students not exerting adequate effort when completing
assessments, resulting in inaccurately low placement in the developmental sequence. Some
students, particularly those who have been out of school for some time, may only need a
refresher of key concepts. It is this group who is often placed into classes that do not match
their abilities, causing additional time and expense to realize their educational goals and
increasing the probability that they will not attain a degree or certificate. Many of these
students report that with a brief review, they would be able to perform better on placement
tests, allowing them to move more quickly through required courses to complete their
education.
8
Implementation of Recommendations:
Task
Time Frame
Create a joint IC/SSC steering committee (include representatives from
WARP, CBS, ARC, FACTC, testing centers) to provide coordination and
ongoing oversight for the implementation process
AugustSeptember
2011
Develop separate working subgroups (recruiting additional expertise from
Fall 2011
key stakeholder groups as needed) to address specific issues related to each
of the main recommendations:
1) Development and distribution process for information about
promising practices around existing innovations with respect to
multiple options for placement
2) Transcript-based placement (include OSPI and other K-12 partners)
3) College readiness profile assessment
4) Placement reciprocity process
5) College processes around placement test orientation and
preparation
[See below for some specifics regarding each of the workgroups.]
1) Development and distribution process for information about
promising practices around existing innovations with respect to
multiple options for placement
By December
2011

Organize and synthesize resources compile during spring 2011 datagathering process
 Develop or utilize existing web-based repository for making resources
available
 Design structured process for colleges to use locally in sharing
resources with appropriate faculty and staff
2) Transcript-based placement (include OSPI and other K-12 partners)


Convene a group of English and math faculty to meet with faculty at
other community and technical colleges who worked with their local
school districts to implement using high school transcripts as part of
the placement process.
Work with the Office of the Superintendent of Public Instruction to
identify courses, along with acceptable course grades and how recent
the courses were taken to determine placement of high school
graduates into a college level English and/or math course, or into a
By December
2011
9
precollege English and/or precollege math course
 Consider whether this approach makes most sense on a local,
regional, or statewide basis.
 Before full implementation, data should be gathered to assess the
effectiveness of using the new process.
3) College readiness profile assessment




Review existing Washington college readiness standards and related
work (e.g., Student Attributes for Math Success project) and develop
specific areas to be addressed by the assessment, including academic
strengths and weaknesses; affective strengths and weaknesses;
motivation, etc.
Evaluate major existing alternatives (e.g., College Student Inventory,
COMPASS and ACCUPLACER diagnostics, etc.) and explore possible
platforms for administering the assessment, including WAMAP.org;
review customization options, validity and reliability for all measures;
conduct fiscal analysis of top alternatives
Resolve feasibility issues and potential implementation issues that
may arise (staffing, cost, time, enrollment timelines, advising)
Implement college readiness assessment on optional basis and collect
data on impact (fall 2013)
4) Placement reciprocity process [NOTE: this work depends on and
follows the work on multiple placement approaches & transcript-based
placement]



Fall 2011-spring
2013
By fall 2012
Include in the 2011-12 work plans of the Instruction and Student
Services commissions the goal of establishing a statewide system
of reciprocity for college-to-college acceptance of student
placement results by fall 2012
Task the Articulation and Transfer Council (ATC) with general
responsibility for crafting a draft reciprocity agreement, following
ATC’s model for negotiating similar agreements in distribution and
diversity requirements, by spring 2012.
Convene a multi-constituent work group to review and comment
on the implementation of the draft agreement in spring-summer
2012, with representation from ATC, ARC, CBS, and the Advising
and Counseling Council.
10
5) College processes around placement test orientation and preparation

Review literature and promising practices identified during datagathering process in spring 2011
 Develop the following resources:
 A script and ideas of content for the assessment orientation video.
 A checklist of components to address in a pre-assessment
orientation process
 Web-based repository for specific resources and/or methods of
assessment test orientation and preparation
Evaluation process:







Create a cross-functional evaluation task force, with WARP as the lead
organization charged with evaluating the new processes and
practices.
Consult with steering committee and relevant workgroups to identify
the new assessment/placement processes and practices for
evaluation.
Gather and review existing published literature regarding the
effectiveness of these processes and practices.
Develop a plan to gather pre/post change data from a representative
sample of colleges. Identify the sources of data and specific analyses
proposed as part of the evaluation. It is anticipated that qualitative as
well as quantitative information will be obtained. Solicit input and
feedback from affected stakeholders across the state regarding the
evaluation plan.
Finalize the evaluation plan, responding to concerns addressed by
stakeholders, including a timeline and resource needs.
After sufficient time for adoption of the new assessment/placement
processes and practices, gather data from the selected colleges and
conduct effectiveness analyses.
Prepare a report summarizing the findings regarding the effectiveness
of the new processes. Identify apparent improvements in the
assessment/placement process as well as any areas of concern
regarding the new system. Develop recommendations for further
improving the system, based on the evaluation results. Share the
results widely across the state with stakeholder groups.
Fall 2011-spring
2012
Fall 2011Summer 2013
Ongoing; varies
based on
timetables for
separate
workgroups
11
Appendix A
Washington Placement Testing Policy Survey Results
(responses from 27 colleges)
1. Few colleges (just under 20%) have a Board policy related to placement testing.
2. No colleges indicated that they have any references in their faculty bargaining agreements to a
faculty role in placement testing.
3. At slightly more half of the colleges, the chief academic officer has the final authority for
changing placement test policies or guidelines as well as for changing the placement test
instruments used, but there are numerous variations across the system on who else is involved
and how that process works.
4. While roughly 2/3 of the colleges responding indicated that the chief academic officer has the
final authority on placement cut score changes, most respondents also indicated that faculty
were heavily involved in the decision.
5. Almost all (25 of 27) colleges accept placement scores from other community and technical
colleges, but only ¼ of the colleges honor placement levels.
6. Just over half (56%) of the colleges publish guidelines for how students can challenge placement
test results.
7. Roughly half of the colleges allow individual math and English departments to employ alternate
assessment tools; of the 14 colleges indicating that they allow alternatives:
a. 6 indicated they used transcripts
b. 10 used writing samples
c. 2 used portfolios
d. 10 used a faculty-designed test
12