PCAT 2012-13
June 27 and 28, 2012
Agenda
•
PCAT Blueprint 2012-2013
•
Updates 2012 Writing Rubric
•
New Norms for 2012 cycle
•
Interpreting PCAT Scores
•
CBT Testing Schedule
•
The Official PCAT Practice Test
•
Reference Links
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
PCAT Blueprint 2012-13
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
Chemistry (approximate % for
each content objective)
PCAT Blueprint 2012-13
Biology Content
2011 –
2012
2012 –
2013
60%
50%
Cellular and Molecular Biology
15%
17%
Diversity of Life Forms
15%
17%
Ecology
5%
––
Plants, Algae and Fungi
5%
––
Animals
10%
––
Health
10%
17%
General Biology
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
Biology (approximate % for each
content objective)
PCAT
Blueprint
2012-13
PCAT Subtest
Part 1: Writing
Part 2: Verbal Ability
Number of Core
Items
Number of
Experimental
Items
1 Prompt (either operational or
experimental)
40
8
Analogies
Sentence Completion
Part 3: Biology
General Biology
Microbiology
25
5
15
40
3
8
20–22
4–5
7–8
1–2
12–13
40
2–3
8
General Chemistry
20–22
4–5
Organic Chemistry
12–13
2–3
7–8
1–2
Human Anatomy & Physiology
Part 4: Chemistry
Basic Biochemistry Processes
Rest Break
Part 5: Writing
Part 6: Reading Comprehension
Copyright © 2012 NCS Pearson, Inc.
1 Prompt (either operational or
experimental)
8 / 1 passage
Comprehension
Analysis
12–13
14–15
2–3
3–4
Evaluation
12–13
2–3
40
8
Basic Math
Algebra
5–6
7–8
1–2
1–2
Probability & Statistics
7–8
1–2
Pre-Calculus
9–10
2–3
Calculus
9–10
2–3
200 multiple-choice
items + 1 writing
prompt
40 multiplechoice items + 1
writing prompt
Total Test
All rights reserved.
30 min.
30 min.
30 min.
30 min.
Variable
40 / 5 passages
Part 7: Quantitative Ability
Time Allowed
30 min.
50 min.
40 min.
240 min. = 4 hrs.
+ Rest Break
Updates 2012 Writing Rubric
•
In an attempt to achieve a more even distribution of writing scores,
Pearson is introducing a new scoring rubric.
•
In 2011, we analyzed data from a pilot scoring of 1,500 essays
from the July PCAT administration using a new 6-point scale.
•
The new 6-point scoring rubric combines elements of the previous
Conventions of Language and Problem Solving rubrics.
•
Results from the pilot data analyses were shared with the PCAT
Advisory Panel, and a decision was made to implement the new
rubric in July 2012.
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
Updates 2012 Writing Rubric
•
The PCAT Writing subtests will be scored using a single rubric that
includes both conventions of language and problem solving
composition criteria.
•
Two scorers will assign a score, and the two scores will be averaged
together for a final score.
•
If the two scores are more than one score point apart,
a resolution score will be averaged with the higher of the
two original scores.
Updates 2012 Writing Rubric
Assigning Scores Based on the PCAT Writing Rubric
•
When scoring candidates’ essays, scorers will consider the
criteria for both conventions of language and problem solving,
and assign a score that meets the criteria for both.
•
Beginning in July 2012, a single Writing score will be reported.
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
Updates 2012 Writing Rubric
•
For example, if an essay meets the conventions of language criteria
for a 6 but only the problem solving criteria for a score of 4, a score
of 4 or 5 may be assigned, depending on the overall quality of the
writing.
•
If an essay is well enough written to earn a 5 or 6 for the
conventions of language criteria and discusses the problem quite
well but fails to provide a solution, the essay be assigned a 3 or 4,
depending on the overall quality of the writing.
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
Updates 2012 Writing Rubric
•
For candidates that seem to have run out of time without providing
a clear conclusion, but that otherwise compose an essay that seems
well enough written, the benefit of the doubt will be given and the
essay will be scored as if a conclusion had been provided.
•
If a candidate argues that the problem stated in the topic is
insolvable or is not really a problem at all, the essay will be scored
on how well this argument is made, even though a solution may not
have been suggested.
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
Updates 2012 Writing Rubric
Score Point 6: Superior
Conventions of Language
•
The writer skillfully applies the conventions of language.
•
The writer makes very few, if any, mistakes in sentence formation, usage, and mechanics,
and none of these errors are serious enough to interfere with the overall flow of the
response or with its meaning.
Problem Solving
• The response exhibits a more sophisticated structural pattern that incorporates a greater
variety of transitional words/phrases and shows some evidence of advanced rhetorical
techniques.
• The response represents a persuasive essay showing strong evidence of efficient
composition skills.
• The solution discussed is clearly related to the problem and is developed with relevant,
convincing support (e.g., facts, examples, anecdotes).
• The main tenets of the problem and the solution are discussed and explained with in-depth
support and detail.
• One or more alternative solutions, or multiple possible solutions, are included with clear
discussion, analysis, and evaluation.
• The response is a logical and effectively-organized argument that is purposefully presented.
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
Updates 2012 Writing Rubric
Score Point 5: Proficient
Conventions of Language
• The writer is proficient in applying the conventions of language.
• Though some mistakes in sentence formation, usage, or mechanics are present, these
errors do not interfere with the overall flow of the response or with its meaning.
• The response proficiently exhibits a structural pattern of multiple paragraphs with a clear
beginning, middle, and end.
Problem Solving
• This response represents a persuasive essay showing evidence of effective compositional
skills.
• The discussion of the problem and solution is clear.
• The solution discussed is clearly related to the problem, and the support presented is
appropriate and relevant, but the response lacks the detailed, in-depth support
characteristic of the highest score point.
• One or more alternative solutions, or multiple possible solutions are discussed, with some
attempt at analysis or evaluation.
• The argument progresses logically with an organizational plan consisting of clear
transitional elements.
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
Updates 2012 Writing Rubric
Score Point 4: Effective
Conventions of Language
•
•
•
The writer is generally successful in applying the conventions of language.
Mistakes in sentence formation, usage, or mechanics are present that may interfere with the
overall flow of the response, but these errors do not interfere with its meaning.
The response exhibits a structural pattern of multiple paragraphs with a beginning, middle,
and end.
Problem Solving
•
•
•
•
This response is generally successful in using important principles of effective composition.
Though the presentation may be general, the discussion of the problem and solution is
reasonably clear.
The solution discussed is generally related to the problem, and most of the support
presented is appropriate and relevant, but the response lacks the depth of support
characteristic of the higher score points.
The argument may be rather loosely organized or may contain digressions in the
organizational structure that lessen the effectiveness of the presentation.
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
Updates 2012 Writing Rubric
Score Point 3: Satisfactory
Conventions of Language
•
•
•
The writer is adequate in applying the conventions of language.
Several mistakes in sentence formation, usage, or mechanics are present that may interfere
with the overall flow of the response and with its meaning.
The response exhibits a structural pattern of multiple paragraphs with elements of a
beginning, middle, and end.
Problem Solving
•
•
•
•
This response is fairly successful in using important principles of effective composition.
Though the presentation may remain too general to be convincing, the discussion of the
problem and solution is adequate.
The solution discussed is adequately related to the problem, and most of the support
presented is appropriate and relevant.
The argument may progress logically, but the loosely organized presentation results in
digressions from the organizational plan or unnecessary redundancies that make the
presentation less straightforward and compromise its effect.
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
Updates 2012 Writing Rubric
Score Point 2: Marginal
Conventions of Language
•
•
•
The writer is marginally successful in applying the conventions of language.
Patterns of mistakes in sentence formation, usage, and mechanics significantly detract from
the presentation.
At times the meaning of the response may be impaired.
Problem Solving
•
The response is marginally successful in using important principles of effective composition.
•
The response may not always exhibit a cohesive structural pattern.
•
•
•
•
The writer may seem more concerned with self-expression than with presenting a logical
argument.
The problem is discussed and a solution related to the problem may be discussed, though
the solution may be either implicit or not clearly stated.
Support is sketchy and, at times, interrupted with redundancies, digressions, irrelevancies,
and/or conditions/qualifications not clearly related to the problem.
Organization of the argument may be rather haphazard, with a loose structuring of ideas
that weakens the effectiveness of the discussion.
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
Updates 2012 Writing Rubric
Score Point 1: Inadequate
Conventions of Language
•
•
The writer’s achievement in applying the conventions of language is limited.
Frequent and serious mistakes in sentence formation, usage, and mechanics make the
response difficult to understand.
Problem Solving
•
•
•
•
•
The response does not successfully embody important principles of effective composition.
It is unclear how the discussion of the problem or solution presented relates to the problem
stated in the prompt.
The support is either fragmentary, unconvincing or is a combination of material that does not
contribute to the presentation (e.g., contradictions, digression, redundancies, and outright
irrelevancies).
Chaotic organization may make it hard to follow the logic of the presentation.
If no solution can be ascertained in the response, then despite any appreciation of the given
problem or how well developed the response is, it cannot exceed to a higher score point.
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
New Norms for 2012 Cycle
•
New percentile rank norms will be introduced in July 2012.
•
The new norms will reflect the performance of a more recent PCAT
examinee population than the previous norms.
•
The new percentile ranks are based on the performance of all firsttime PCAT examinees who took the test from June 2007 through
January 2011.
•
The scaled scores will not change.
•
A new Technical Manual with detailed information about the norms
was recently sent to all pharmacy programs to use with the July
2012 scores.
•
Additional downloadable documents useful in interpreting PCAT
scores are available on the PCAT website (PCATweb.info).
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
Interpreting PCAT Scores
Score Reporting
•
An Official Score Report is available to each candidate online at
the PCAT website by logging in to the candidate’s profile, with
email alerts sent to candidates when the reports are available.
•
Official Transcripts, showing all PCAT scores earned within the
previous five years, are sent to institutions, either printed (via
mail) or electronically (via SFTP for Pearson subscribers or via
PharmCAS).
•
Pharmacy schools receiving paper transcripts receive a
summary of all candidates whose scores were sent to their
institution.
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
Interpreting PCAT Scores
PCAT personal Score Reports and Official Transcripts present
three types of scores:
• Scaled Scores
• Percentile Ranks
• Writing Scores
Neither Pearson nor the AACP establishes passing scores for
individual PCAT subtests or for the PCAT as a whole. It is the
responsibility of each school of pharmacy to determine how it
uses PCAT scores.
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
Interpreting PCAT Scores
Scaled Scores
•
Range from 200-600, making scores directly comparable
across test forms and from year to year.
•
Reported for the five multiple-choice subtests and
Composite score.
Percentile Ranks
•
Range from 1-99.
•
Indicate a candidate’s performance relative to the current
PCAT normative sample (all first-time examinees from
June 2007 through January 2011).
•
Are provided for each of the five multiple-choice subtests
and the Composite score.
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
Interpreting PCAT Scores
Writing Scores
•
Range from 1.0-6.0 with 6.0 being the highest earned
score possible.
•
Essays will be scored using a single rubric that included
conventions of language and problem solving
composition criteria.
•
The Writing score represents an average of scores
assigned by two scorers.
For example, a score of 4.5 is the average of one scorer
assigning a 4.0 and another scorer assigning a 5.0.
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
Interpreting PCAT Scores
Mean Writing Scores
•
•
The mean Writing scores differ for each PCAT test
administration, reflecting:
•
differences in prompt content
•
slight differences in prompt difficulty, and
•
differences in the group of candidates sitting
for the PCAT during testing window.
For this reason, a candidate’s Writing scores should
always be evaluated relative to the mean scores for
the test date on which the scores are earned.
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
Interpreting PCAT Scores
Official Transcript Sample
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
Interpreting PCAT Scores
Reporting Personal Information
•
Pearson uses only the last four SSN digits to identify a
candidate to reduce candidate exposure to identity theft in the
event that data are compromised in any way.
•
The PCAT CID (Candidate ID) is a unique identifier and is
suggested to be used to match PCAT candidates.
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
Reliability and Validity
Reliability
•
Satisfactory internal consistency for multiple-choice subtest and
Composite scores (K-R 20 = 0.82–0.95)
•
High inter-rater reliability for Writing scores (over 0.99 for 0–1 point
discrepancy)
Internal Structure Validity
• Intercorrelations between PCAT subtest and Composite scores (2011-12)
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
Reliability and Validity
Correlations Between Predictors and GPA in First Year of
Pharmacy Program (N = 2,244)
Predictive Validity
• The results of this Pearson study
are comparable in many ways to
previous research findings that
have shown the moderate
validity of PCAT scores in
predicting GPAs during the first
year of pharmacy school.
• In the most comprehensive PCAT
study conducted to date, Kuncel,
Crede, Thomas, Kleiger, & Seiler
found in a meta-analysis of
previous research that both
PCAT scores and pre-pharmacy
GPAs were valid predictors of
student success during the first
3 years of pharmacy school (Am
J Pharm Educ.
2005;69(3):Article51).
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
First Year GPA
Uncorrected
Corrected
r
r*
PCAT Verbal Ability SS
PCAT Biology SS
0.13
0.26
0.16
0.35
PCAT Reading Comprehension SS
0.17
0.24
PCAT Quantitative Ability SS
0.21
0.28
PCAT Chemistry SS
0.29
0.36
PCAT Composite SS
0.32
0.44
PCAT Writing: Conventions of Language
0.06
PCAT Writing: Problem Solving
0.06
Entering GPA: Cumulative
0.44
Entering GPA: Math
0.28
Entering GPA: Science
0.44
Variable
From: Meagher, D. G., Pan, T., & Perez, C. P. (2011). Predicting performance in the
first-year of pharmacy school. American Journal of Pharmaceutical Education,
75(5), Article 81.
Abbreviations: SS = Scaled Score; GPA = grade point average; r = correlation
coefficient; r2 = the percent of one variable explainable by another variable
All correlations are weighted means of the correlations for the 22 schools combined
based on Fisher's z transformations; the corrected correlations were done using
the formula of Cohen, et al. (2003, p. 58).
The Official PCAT Practice Test
Three Official PCAT Practice Tests are available at PCATweb.info.
Examinees who took the Official PCAT Practice Test:
•
Average higher on the
actual PCAT compared to those
who have not taken the Official
PCAT Practice Test.
•
Show significant score increases
on their 2nd PCAT attempt
after taking the Practice Test.
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
The Official PCAT Practice Test
The Official PCAT Practice Test
includes a Diagnostic Report
that:
•
assesses performance and
•
evaluates strengths and
weaknesses by suggesting areas
that need review.
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
Reference Links
• YouTube videos at
www.youtube.com/PearsonFieldResearch
• Up-to-date and comprehensive information found in the
downloadable Candidate Information Booklet (CIB) at
PCATweb.info, which is also available as a printed booklet at
pharmacy schools and admissions offices.
• A CBT testing tutorial is available at
PEARSONVUE.com/PCAT
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
Reference Links
• PCAT Technical Manual and PCAT Score Interpretation
Quick Reference
www.aacp.org/resources/academicpolicies/admissio
nsguidelines/Pages/PharmacyCollegeAdmissionsTest
.aspx
• PCAT Website
www.PCATweb.info
• PCAT Validity Study published in AJPE
www.aacp.org/resources/academicpolicies/admissio
nsguidelines/Documents/PCATPredictiveValidityStud
y--GeneralReport.pdf
www.ajpe.org/aj7sos/aj750581.pdf
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
Reference Links
• PCAT Basics
• Interpreting PCAT Scores
• PCAT Reliability and Validity
http://pcatweb.info/PCAT-Information-forAdminstrators-and-Faculty.php
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
Questions?
Copyright © 2012 NCS Pearson, Inc.
All rights reserved.
Contacts:
Cindy Agonis
Cindy.Agonis@Pearson.com
210-339-5302
Don Meagher
Don.Meagher@Pearson.com
210-339-5297