PCAT: Proposed Changes for
the Future
Paul W. Jungnickel, Ph.D., R.Ph.
Renee M. DeHart, Pharm.D.
AACP Governance
Admissions
PCAT Advisory Committee
PharmCAS Advisory Panel
Gayle Brazeau, Univ of New Engl
Jennifer Adams, AACP
Kathy Besinque, Univ Southern Cal
Renee DeHart, Samford University
Mary Euler, Univ of Charleston
Eric Hanson, Roseman University
Paul Jungnickel, Auburn University
Rodney Larson, Husson University
Paul Ranelli, Univ of Minnesota
Theodore Tong, Univ of Arizona
Jennifer Clutter, West Virginia Univ
Jennifer Adams, AACP
Tom TenHoeve, Univ of Illinois
Jeremy Altschafl, Univ Wisconsin
Renae Chestnut, Drake University
Joel Gonzales, Univ California-SF
Peter Haeg, University of Minnesota
Jennifer Hess, University of Buffalo
Mike Kelly, University of Iowa
Jonathan Parker, Samford Univ
Cyndi Porter, Calif Northstate Univ
Andrea Wall, Univ of Cincinnati
AACP Interim meeting
February 2013
Special meeting of the PCAT Advisory
Committee included members of PCAT
Advisory Committee, PharmCAS Advisory
Panel, Peggy Piascik, President-elect
AACP, Kathy Kelley, Marilyn Speedie,
Pearson staff
Argus Commission 2012
Recommendation of the Argus Commission 2012 report
Examination of the Role of Pre-Pharmacy
Education in Cultivating “Habits of the Mind” in
Pharmacists
Recommendation : use of assessment tools for affective
domain- critical thinking, problem solving,
inquisitiveness, professionalism
Cultivating ‘Habits of Mind’ in the Scholarly Pharmacy Clinician: Report of the 2011-12 Argus Commission AJPE 2012:76(6)
PCAT
Can PCAT be modified to assess the
affective domain?
Moving Forward: Proposed
Revisions to the PCAT
• Present current PCAT model and
proposals for 2 additional years
• Highlight changes under consideration
– Greater critical thinking and reasoning
– Items based on more extensive reading
– Decreased test duration
• Solicit discussion and feedback
Current PCAT Blueprint
Number of
Number of
Operational
Experimental
PCAT Subtest
Items
Items
Part 1: Writing
1 Prompt (operational/experimental)
Part 2: Verbal Ability
40
8
Analogies
25
5
Sentence Completion
15
3
Part 3: Biology
40
8
General Biology
20–22
4
Microbiology
7–8
1–2
Anatomy & Physiology
12–13
2–3
Part 4: Chemistry
40
8
General Chemistry
20–22
4–5
Organic Chemistry
12–13
2–3
Basic Biochemistry Processes
7–8
1–2
Rest Break
Part 5: Writing
1 Prompt (operational/experimental)
Part 6: Reading Comprehension
Comprehension
Analysis
Evaluation
Part 7: Quantitative Ability
Basic Math
Algebra
Probability & Statistics
Pre-Calculus
Calculus
Total Test
40 / 5 passages
12–13
15
12–13
40
5–6
7–8
7–8
9–10
9–10
200 multiplechoice items + 1
writing prompt
8 / 1 passage
2–3
3–4
2–3
8
1–2
1–2
1–2
2–3
2–3
Time Allowed
30 min.
30 min.
30 min.
30 min.
15 min.
30 min.
50 min.
40 min.
40 multiple-choice
240 min. = 4 hrs.
items + 1 writing
+ Rest Break
prompt
Options for 2014-2015
• Goal is to move towards items that require higher order
thinking and reasoning
• Field-test items for Biology, Chemistry, and with more
extensive passages (4 items per passage)
• Field-test Quantitative items with problem or scenario
stated in stem text
• Field-test Reading Comprehension items with humanities
and Social Science content
Sample Biology Passage: Chronic Stress and Immunity
Stress is an intrinsic part of life for most organisms, and dealing successfully with stressors enables
survival. Early studies in rats showed that acute stress results in a redistribution of leukocytes from the
blood to other organs such as skin, lymph nodes, and bone marrow, and that adrenal stress hormones are
the major mediators of this response. Because the skin is one of the targets to which leukocytes traffic
during stress, researchers hypothesized that such a leukocyte redistribution may increase immune
surveillance and consequently enhance immune function if skin is exposed to an antigen after acute stress.
To test this hypothesis, researchers examined the effects of acute stress on skin immunity by
exposing rats to containment in a restraining cage (see partial results below in Table 1). Physical restraint
activates the sympathetic nervous system and the hypothalamic-pituitary-adrenal axis, and results in the
activation of adrenal steroid receptors throughout the body. Results showed larger numbers of leukocytes
in the skin of stressed animals both before and after exposure to an antigen. In contrast to the
immunoenhancing effects of acute stress, researchers found that chronic stress significantly suppressed
the immune response. An adrenalectomy eliminated the stress-induced enhancement of skin immune
response.
Table 1: Physiological Change from Baseline in Response to Stimuli
Stimulus
A
B
C
D
Heart
Rate
Increase
Increase
Decrease
Decrease
Bronchiole
Diameter
Decrease
Increase
Increase
Decrease
Bone Marrow
Leukocytes
Increase
Increase
Decrease
Decrease
Sample Biology Items for Passage: Chronic Stress and Immunity
Sample Item 1
According to the passage, which of the following is likely to act as a mediator for the redistribution of
blood to other organs in response to an acute stressor?
A. Aldosterone
B. Glucagon
C. Cortisol
D. Vasopressin
Correct Answer: C
Sample Item 2
If the study described in the passage had examined the effects of an acute stressor on digestion, the
information in the passage would best support the prediction that
A. peristalsis would be increased.
B. anal sphincter tone would be decreased.
C. salivation would be increased.
D. intestinal motility would be decreased.
Correct Answer: D
Sample Biology Items for Passage: Chronic Stress and Immunity
Sample Item 3
Which of the following therapies is designed to harness the physiological response to stress that
researchers described in this study?
A. Exercise prior to the administration of a vaccine
B. Antibiotic treatment for a skin infection
C. Relaxation meditation before a chemotherapy session
D. Physical therapy for an atrophied muscle
Correct Answer: A
Sample Item 4
According to information in the passage, which stimulus in Table 1 most likely represents an acute
stressor?
A. A
B. B
C. C
D. D
Correct Answer: B
Proposed PCAT Test Blueprint for 2014–15: Field-Testing New Item Types
PCAT Subtest
Part 1: Writing
Part 2: Verbal Ability
Analogies
Sentence Completion
Part 3: Biology
General Biology
Microbiology
Anatomy & Physiology
Part 4: Chemistry
General Chemistry
Organic Chemistry
Basic Biochemistry Processes
Part 5: Reading Comprehension
Comprehension
Analysis
Evaluation
Part 6: Quantitative Ability
Basic Math
Algebra
Probability & Statistics
Pre-Calculus
Calculus
Total Test
Number of
Number of Experimental
Operational Items
Items / Passages
1 Prompt (operational)
40
0
25
0
15
0
8 / 2 (experiment, scenario, or
40
other context in passage)
20–22
4
7–8
1–2
12–13
2–3
8 / 2 (experiment, scenario, or
40
other context in passage)
20–22
4–5
12–13
2–3
7–8
Rest Break
40 / 5 passages
12–13
15
12–13
40
5–6
7–8
7–8
9–10
9–10
200 multiple-choice
items + 1 writing
prompt
Time Allowed
30 min.
25 min.
35 min.
35 min.
1–2
15 min.
8 / 1 (humanities or social
sciences content)
2–3
3–4
2–3
8 (problem or scenario stated
in stem text)
1–2
1–2
1–2
1–2
1
32 multiple-choice items
50 min.
45 min.
220 min. = 3 hrs.
40 min. + Rest
Break
Looking Forward to
2015-2016
• Goal is to increase testing of critical thinking
• Biological and Chemical processes: 40-50 percent of
items with passages
• Critical Reading: 40-60 percent of items from
humanities and social sciences
• Quantitative Reasoning
– 40-50 percent of items with stem text set-up
• Less emphasis on calculus and pre-calculus
• Remove Verbal Ability – need your feedback
– Would allow more time to test new item types in
Biology, Chemistry and Quantitative sections
Correlations (r) Between PCAT Verbal Ability and
Reading Comprehension Scaled Scores
r
n
Normative
Sample:
2007–11
0.69
68,241
All Cases:
2012–13
0.67
27,753
Note. Correlations significant at <.0001 level. Normative Sample = First
attempts only.
PCAT Writing Score Point Distributions
(Mean = 3.27): July 2012 through January 2013
1.0
1.5
2.0
4.0
4.5
5.0
5.5
6.0
n
213
204 2,955 2,301 7,574 5,339 8,291
597
182
10
42
%
0.8
0.7
2.2
0.7
0.0
0.2
10.7
2.5
8.3
3.0
27.3
3.5
19.3
29.9
Note. Only valid scores are displayed (no 0 scores).
Proposed PCAT Test Blueprint for 2015–16: Introducing New Operational Items
Operational Experiment
Items /
al Items /
Time
PCAT Subtest
Passages
Passages
Allowed
Part 1: Writing
1 Prompt (operational)
30 min.
Part 2: Biological Processes
40 Items / 4–5 8 Items / 2
(40–50% of items with passages)
Passages
Passages
40 min.
General Biology
20–22
4
Microbiology
7–8
1–2
Anatomy & Physiology
12–13
2–3
Part 3: Chemical Processes
40 Items / 4–5 8 Items / 2
(40–50% of items with passages)
Passages
Passages
40 min.
General Chemistry
20–22
4–5
Organic Chemistry
12–13
2–3
Basic Biochemistry Processes
7–8
1–2
Rest Break
15 min.
Part 4: Critical Reading
(40–60% of items with humanities
40 Items / 5
8 Items / 1
or social science passages)
Passages
Passage
50 min.
Comprehension
12–13
2–3
Analysis
15
3–4
Evaluation
12–13
2–3
Part 5: Quantitative Reasoning
(40–50% of items with stem text
set-up)
40
8
45 min.
Basic Math
9–10
1–2
Algebra
9–10
1–2
Probability & Statistics
7–8
1–2
Pre-Calculus
7–8
1–2
Calculus
5–6
1
160 multiple32 multiple- 205 min. = 3
choice items
hrs. 25 min.
Total Test
choice
+ 1 writing
+ Rest Break
items
prompt
Situational Judgement Testing
Argus Commission 2012
Recommendation of the Argus Commission 2012 report
Examination of the Role of Pre-Pharmacy
Education in Cultivating “Habits of the Mind” in
Pharmacists
Recommendation : use of assessment tools for affective
domain- critical thinking, problem solving,
inquisitiveness, professionalism
Cultivating ‘Habits of Mind’ in the Scholarly Pharmacy Clinician: Report of the 2011-12 Argus Commission AJPE 2012:76(6)
Desired SJT Assessments
Presented to Pearson
•
•
•
•
•
•
Empathy and altruism
Ethical reasoning, integrity
Interpersonal skills
Team player
Self awareness
Problem solving, creativity
Situational Judgement Tests (SJTs)
• Test which presents realistic,
hypothetical scenarios
• SJTs not simply “one size fits all” tests,
but are designed to address specific
role requirements
• Have been used in functions such as
personnel selection
Sample SJT question
You and a co-worker are working on a complex project that demands a great deal of effort
from both of you. Your co-worker is frequently absent as a result of stress due to personal
problems. You have not known him for long and know little about the circumstances. Your
co-worker contributes very little to the project, and as a result, you are putting in an
excessive amount of overtime in order to keep the project moving ahead.
Rank in order the following actions in response to this situation
(1= Most appropriate; 5= Least appropriate).
A.
B.
C.
D.
E.
Ask other co-workers to help you manage your workload.
Request additional help from your manager to ensure that the project is completed on
schedule.
Meet with your co-worker to request that he does his share of the work.
Continue to put in overtime to keep the project moving ahead.
Offer to help your co-worker deal with his personal problems.
Mark your ranking for each option by circling the appropriate number on the answer sheet.
For example: If you thought option A was the most appropriate in response to Question 1, B
the second most appropriate, C the third, D the fourth and E the least appropriate option, you
would complete the answer sheet as follows (please note that you may not give 2 options
the same rank):
http://www.pearsonvue.com/nphstr/situational_judgement_practice.pdf
Sample SJT question
http://www.thelancetstudent.com/legacy/category/articles/medical-eductation/
Sample Scoring
http://www.foundationprogrammeguru.co.uk/the-situational-judgement-test.html
Typical SJT characteristics
• Scenarios are role-driven
• Select options are presented
• Participants must select from provided
options
• Scoring rubrics
Comparison of SJTs and Various
Interviewing Techniques
SJT: Current Status
• Pearson’s Talent Team is further
researching SJTs
• AAMC also looking at SJTs
Discussion and Questions