Using Participatory Action Research to
Develop and Validate the Core
Competency Measure (CCM)
Stephen S. Leff, Ph.D., Nathan Blum, M.D., Abbas Jawad, Ph.D.,
Judith Silver, Ph.D., Symme Trachtenberg, MSW, Karen Hudson,
MSW, & Karen Tate, Family Faculty
The Children’s Hospital of Philadelphia & University of Pennsylvania School of Medicine
Poster presented at the Association of University Centers on Disabilities Annual Meeting
and Conference (Washington, DC, November, 2007)
Requests for reprints: Stephen S. Leff; Children’s Hospital of Philadelphia,
Dept. of Psychology, Rm. 1480 at 3535 Market, 3405 Civic Center Blvd.;
Philadelphia, PA 19104; e-mail: leff@email.chop.edu
Purpose and Overview

The LEND Program Evaluation Committee at CHOP has
developed specific measurement tools to assess the
impact of LEND fellowship training

We will highlight the Core Competency Measure (CCM)
– The CCM assesses fellows’ perception of their knowledge and
skills across core competency domains of LEND

We will discuss how the CCM was designed through a
participatory action research model

We will examine an initial validation sample for the past
five cohorts of LEND fellows, and discuss potential
implications and uses of this new measure
The LEND Program Evaluation
Committee at CHOP

Interdisciplinary Team including:
–
–
–
–
–

Psychologists
Biostatistician
Pediatrician
Social Worker
LEND fellows from a variety of disciplines
Primary Goals
– Develop a core set of outcome measures
– Modify LEND curriculum based upon outcome evaluation
(translating research to practice)
– Provide mentoring to fellows in conducting program
evaluations through a Participatory Research model
Participatory Action Research
(PAR) Framework
Combining theoretical principles, past
empirical research, and key stakeholder
input into design of measures
(Leff et al., 2004)
 Partnership drives the project
 Researchers & participants form nonhierarchical trusting relationships
 Partnership ensures meaningfulness
 Links Research to Practice

Description of Measures

Core Competency Measure (CCM)
– Modeled after the core dimensions of LEND training
– Originally piloted with two cohorts of LEND fellows
– Addresses LEND’s 6 Major Objectives
•
•
•
•
•
•
Clinical Skills (8 items)
Interdisciplinary Collaboration (5 items)
Family-Centered & Culturally-Sensitive Services (7 items)
Community-Based Services (9 items)
Research Methods (9 items)
Knowledge of Disability Policies & Regulations (6 items)
– Likert response scale
• 1=Disagree, 2=Somewhat Agree, 3=Agree; 4=Strongly Agree
Subscales of the CCM
Sample Items

Clinical Skills
– I have a good understanding of normal development
– I am able to identify common procedures for early screening &
diagnosis of developmental delay or disability
– I am able to discuss the impact of disability on the child & family

Interdisciplinary Collaboration
– I have knowledge of the different characteristics of health care
professions, including assessment tools, developmental domains, an
intervention strategies addressed by the respective disciplines
– I understand ways to foster strong collaborations with health care
professionals from different disciplines

Family-Centered & Culturally-Sensitive Services
– I am knowledgeable about the key elements of family-centered care &
its impact on care delivery outcomes
– I can describe a typical daily routine & activities families may encounter
when living with a special needs child
– I am knowledgeable about how culture & traditional/non-traditional
beliefs about health care impact families’ experiences with health care
delivery
Subscales of the CCM
Sample Items

Community-Based Services
– I am able to refer and coordinate community-based care services for
children with special needs
– I can collaborate well with colleagues from community-based agencies
and families to develop transition plans
– I have a good understanding of how to influence practice in a broad
way via institutional and systems change

Evaluate and Contribute to Research
– I am able to easily develop research questions
– I understand the role of partnership in designing, implementing, and
evaluating research projects
– I have an understanding of the role of the Institutional Review Board
and how to prepare an IRB proposal

Knowledge of Regulations/Entitlements that Impact
Youth with Disabilities
– I am able to effectively advocate to support child & family entitlements
– I can critically examine examples of discrimination towards people with
disabilities or special health care needs
Using PAR to Design the CCM

The CCM was designed by combining what
was considered the core dimensions of LEND
with literature reviews on each primary domain
combined with feedback from our Program
Evaluation Committee

A draft version of the CCM was then piloted with
two cohorts of LEND fellows at the beginning
and end of their respective training years

The Program Evaluation Committee and LEND
fellows provided additional feedback on
subscales, wording, and likert-response scales

The CCM measure was then finalized
Psychometric Analyses
Subscales
Internal Consistency Test-Retest Reliability
(n = 73)
(n = 21)
Clinical
 = .82
r = .82
Interdisciplinary
 = .78
r = .62
Family & Culture
 = .88
r = .72
CommunityBased Services
 = .91
r = .85
Research
 = .94
r = .94
Regulations &
Entitlements
 = .87
r = .80
Intercorrelations Between Subscales (n=73)
Clinical
Clinical
Interdisc.
Family/
Culture
Community
Research
Regulations
1
Interdisc. Family/Cult. Community Research Regulations
r=.54**
r=.28*
r=.16
r=.22
r=.17
1
r=.59**
r=.51**
r=.20
r=.42**
1
r=.60**
r=.19
r=.51**
1
r=.19
r=.63**
1
r=.08
1
Core Competency Measure (CCM)
Pre- and Post-LEND Year Results
4
3.5
3
2.5
2
1.5
1
Pre
lin
ica
er
l
di
sc
ip
lin
Fa
ar
y
m
ily
-C
ul
tu
re
C
om
m
un
ity
R
es
ea
rc
h
R
eg
ul
at
io
ns
Post
In
t
C
Mean
4 Cohorts of LEND Fellows (N = 53)
Goals
-- Significant
differences found
for each subscale;
-- p < .0001;
Effect Size range
from 0.84 to 1.20
(“large effects”)
Summary

Summary
– Use of a Participatory Action Research model helps to
ensure the usefulness of measures, and has allowed
us to modify our curriculum to be responsive to
fellows’ feedback
– The CCM has strong internal consistency and
adequate test-retest reliability across the six subscales
– The Interdisciplinary, Family-Culture, Community, &
Regulations/Entitlements subscales are highly
intercorrelated, while the Research subscale is not
significantly correlated with any other subscales
– The CCM appears to be sensitive to treatment effects.
• Overall, LEND fellows clearly report significant
knowledge/expertise advances in all areas over the course of
the LEND year (e.g., clinical, interdisciplinary, family/cultural,
community, research, and disability regulations/entitlements)
Future Directions
– The CCM could be used to help other LEND
and UCEDD Programs evaluate whether their
fellows make progress in these six core
competency areas over the course of their
training
– We are currently assessing how well the CCM
correlates with other program evaluation
measures we have designed and validated
related to Research Competency and
Interdisciplinary Collaborations