Assessing the
Development of Critical
Thinking Skills in
Undergraduate
Psychology Students
Phil Dunwoody & Mark McKellop
Partially funded by grants from the Teagle Foundation, the Council of Independent
Colleges/Collegiate Learning Assessment Consortium, and the James J. Lakso Center for the
Scholarship of Teaching & Learning at Juniata College.
What is Critical Thinking?

American Philosophical Association’s
“Delphi Report”
(Facione, 1990)
 CT is “purposeful, self-regulatory judgment which
results in interpretation, analysis, evaluation, and
inference, as well as explanation of the evidential,
conceptual, methodological, criteriological, or
contextual considerations upon which that
judgment is based.”
What is Critical Thinking?

CT is multidimensional construct – it is
both a disposition & a learned skill
(Halpern, 1998, 2003)

CT has general & discipline-specific
aspects with limited generalizability
(Lehman & Nisbett, 1990; Toplak & Stanovich, 2002; Williams et al., 2004)
What is Critical Thinking?

American Psychological Association
“Guidelines for the Undergraduate
Psychology Major” (APA, 2007)
◦ Goal 3: Critical Thinking Skills in Psychology.
 Students will respect and use critical and creative
thinking, skeptical inquiry, and, when possible,
the scientific approach to solve problems related
to behavior and mental processes.

Documenting & Assessing our Teaching
◦ Departmental Assessment Plan
◦ Mapping our POE
Psychological Critical Thinking (PCT)

PCT is “the ability to judge the plausibility of
specific assertions, to weigh evidence, to assess
the logical soundness of inferences, to construct
counter-arguments and alternative hypotheses.”
(Nickerson et al, 1985)

CT skills in psychology include:
◦
◦
◦
◦
◦
Argument analysis and evaluation
Methodological reasoning
Statistical reasoning
Causal reasoning
Focusing and clarifying questions
(Bensley & Murtaugh, 2012)
Psychological Critical Thinking

Our assessment/measure of PCT will
◦ be relatively brief
◦ be (relatively) easily scored
◦ evaluate “applied” critical thinking skills
(can one “think like a behavioral scientist?”)

PCTI focuses on following aspects of PCT:
 Appropriate evaluation of expert behavioral research
(i.e., peer-reviewed journal articles)
 Appropriate evaluation of non-expert behavioral claims
(i.e., media reports)
 Identifying reasoning faults in faux-novice designed
behavioral research
Psychological Critical Thinking
Inventory (PCTI)






California Critical Thinking Skills Test
Watson-Glaser Critical Thinking Appraisal
Lawson’s Psychological Critical Thinking
Exam
Collegiate Learning Assessment
Psychology Area Concentration
Achievement Test (PACAT)
National Survey of Student Engagement
(NSSE)
Validating the PCTI

Participants
◦ Intro Psych: 96 (Spring, 2011)
◦ Senior Psych Capstone: 39 (Spring & Fall, 2011
and Spring, 2012)
◦ Pre and Post Research Methods: 29 (Fall, 2011
and Spring, 2012)
 Pre and Post with CAT currently
Participants

Popular Press

Critique Faux-Novice Study

Peer-review Articles

Total PCTI
◦ Possible range: 0-10 points
◦ Observed range=2-8, Mean=5.9, SD=1.7
◦ Possible range: 0-7 points
◦ Observed range= 0-6, Mean=2.9, SD=1.3
◦ Possible range: 0-7 points
◦ Observed range= 0-6, Mean=3.28, SD=1.3
◦ Possible range: 0-24
◦ Observed range= 6-20.5, Mean=12.1, SD=2.7
Descriptive Stats PCTI
Graduating psychology seniors will score higher on PCTI
than freshman in Introductory Psychology.
 Popular Press

◦ Intro:
M=5.19, SD=1.48
◦ Capstone:
M=6.55, SD=1.99
◦ Significant: t(91)=-3.72, p<0.001, d=0.78

Critique Faux-Novice Study

Peer-review Articles

Total PCTI
◦ Intro:
M=2.48, SD=1.19
◦ Capstone:
M=3.88, SD=1.26
◦ Significant: t(91)=-5.27, p<0.001, d=1.14
◦ Intro:
M=3.03, SD=1.37
◦ Capstone:
M=3.34, SD=1.45
◦ Significant: t(91)=-1.02, p=0.31
◦ Intro:
M=10.70, SD=2.34
◦ Capstone:
M=13.77, SD=2.76
◦ Significant: t(91)=-5.64, p<0.001, d=1.20
Hypothesis 1
Students in Research Methods should score higher at the
end of the semester than at the beginning of the semester
(alternate form).
 Popular Press

◦ Pre:
M=6.52, SD=1.46
◦ Post:
M=7.30, SD=1.32
◦ Significant: t(26)=-2.09, p=0.05, d=0.56

Critique Faux-Novice Study

Peer-review Articles

Total PCTI
◦ Pre:
M=3.26, SD=1.10
◦ Post:
M=3.81, SD=1.27
◦ Significant: t(26)=-1.86, p=0.07, d=0.46
◦ Pre:
M=3.67, SD=1.30
◦ Post:
M=3.93, SD=1.75
◦ Significant: t(26)=-0.55, p=0.59
◦ Pre:
M=13.44, SD=2.68
◦ Post:
M=15.04, SD=2.49
◦ Significant: t(26)=-2.39, p=0.02, d=0.62
Hypothesis 2
PCTI Convergent Validity
PCTI with measures of generic
(abstract) critical thinking
PCTI with Lawson’s Psychological
Critical Thinking Exam
PCTI with measures of educational
achievement
PCTI with measures of Psychology
Knowledge (PACAT)






Our PCTI has good face validity and shows
clear differences not only between Freshman
and Seniors, but also within one course.
The PCTI does not correlate much with more
traditional abstract measures of CT.
The PCTI clearly correlates with the
psychology knowledge-based PACAT.
The PCTI also clearly correlates with general
measures of academic achievement such as
GPA, SAT V, and SAT M.
Intelligence tests also correlate with these
same measures.
So…
Conclusion

"[Intelligence is] the aggregate, or global
capacity to act purposefully, think
rationally, and deal effectively with the
environment. It is global because it
characterizes the individual's behavior as
a whole; it is an aggregate because it is
composed of elements or abilities which,
though not entirely independent, are
qualitatively differentiable.”
◦ David Wechsler, 1938
Is critical thinking just another
term for intelligence?









American Psychological Association. (2007). APA guidelines for the undergraduate
psychology major. Washington, DC: Author. Retrieved from
www.apa.org/ed/resources.html
Bensley, D. A., & Murtagh, M. P. (2012). Guidelines for a scientific approach to
critical thinking assessment. Teaching of Psychology, 39, 5-16.
Facione, P. A. (1990). Critical thinking: A statement of expert consensus for
purposes of educational assessment and instruction. Millbrae, CA: The California
Academic Press.
Halpern, D. F. (1998). Teaching critical thinking for transfer across domains:
Dispositions, skills, structure training, and metacognitive monitoring. American
Psychologist, 53, 449–455.
Halpern, D. F. (2003). Thought and knowledge: An introduction to critical thinking.
Mahwah, NJ: Erlbaum.
Lehman, D. R., & Nisbett, R. E. (1990). A longitudinal study of the effects of
undergraduate training on reasoning. Developmental Psychology, 26, 952-960.
Nickerson, R. S., Perkins, D. N., & Smith, E. E. (1985). The teaching of thinking.
Hillsdale, NJ: Lawrence Erlbaum.
Toplak, M. E., & Stanovich, K. E. (2002). The domain specificity and generality of
disjunctive reasoning: Searching for a generalizable critical thinking skill. Journal of
Educational Psychology, 94, 197-209.
Williams, R. L., Oliver, R, & Stockdale, S. (2004). Psychological versus generic critical
thinking as predictors and outcome measures in a large undergraduate human
development course. The Journal of General Education, 53, 37-58.
References