Taking Stock and Moving Forward:
HPPAE Measuring Student Outcomes
JoAnn Damron Rodriguez, UCLA
Noell Rowan, University of North Carolina,
Wilmington &
Julie Birkenmaier, Saint Louis University,
October 3rd, 2013
Taking Stock and Moving Forward:
A series of three webinars looking at:
 What’s been learned from HPPAE evaluation data,
What questions have been addressed in published articles,
What further research questions to consider
and How the instruments and data can be used
What we can Learn from HPPAE Graduates
Date: Thursday, October 24th, 2013, 2:00 EST
Facilitators: Emily Greenfield, Rutgers University, Sang Rachel Lee, San Jose State
University & Bert Chantarat, Center for Evaluation and Applied Research (CEAR) at
NYAM
HPPAE Institutional Outcomes and Implications for Policy
Date: Thursday, November 7th, 2013, 2:00 EST
Facilitators: Vicky Rizzo, Binghamton University, Pat Volland, Social Work Leadership
Institute at Silberman School of Social Work at Hunter College & Linda Weiss, CEAR at
NYAM
www.socialworkleadership.org
HPPAE
Measuring Student Outcomes
1. Instruments Developed and Tested in the HPPAE
Collaborative Evaluation
2. Exemplary Articles Published Using the National
HPPAE Dataset (See HPPAE Publication
Bibliography)
3. Future Research Questions for Social Work
Education Using the Instruments and/or the
HPPAE Dataset
HPPAE Instruments:
Measuring Student Outcomes

HPPAE STUDENT PRE-TEST








HPPAE STUDENT POST-TEST





A. CAREER INTEREST (2RD webinar)
B1. REVISED KNOWLEDGE OF AGING FOR GERIATRIC SOCIAL WORK QUIZ
B2. GERONTOLOGICAL PRACTICE-BASED SOCIAL WORK QUIZ
C. KNOWLEDGE OF SERVICES FOR OLDER ADULTS
D. HPPAE GERIATRIC SOCIAL WORK COMPETENCY SCALE II
E. FIELD INSTRUCTOR VERSION COMPETENCY SCALE II
F. BACKGROUND INFORMATION
A. BACKGROUND INFORMATION
B. All THE ABOVE KNOWLEDGE AND COMPETENCE MEASURES
C. CAREER INTEREST (2RD webinar)
CAREER TRACKING (2rd webinar)
INSTITUTIONAL LEVEL DATA (3RD webinar)
Background Variables
 VARIABLES
 Academic
 Experience with Older Adults
 Demographic
 Measured with some variation Pre & Post
 Measured for students & field instructors (Pre)
B1. REVISED KNOWLEDGE OF AGING
FOR GERIATRIC SOCIAL WORK QUIZ
 Developed to assess knowledge of aging as an
outcome of gero social work education
 Selected 25 items from versions of Facts of
Aging Quiz (Palmore, 1998) based on
biopsychosocial framework
 Multiple choice items
 Score range from 0 to 25
Findings KASW Summary
1. KASW was strongly correlated with students’
aging-related educational experiences
2. KASW was strongly correlated with students’ selfrated geriatric social work competency
3. Psychological and social items in KASW were
more strongly correlated than to the biological
ones
4. Analysis demonstrated good reliability,
Cronbach’s Alpha (α) = .84
5. Language Palmore outdated, some items
markedly weaker, need for revision
Examples Revised KASW
6. With age, intelligence:
a. Declines for all types of intelligence
b. Decreases in some domains and
increases in others
c. Increases until age 70, and then
decreases
d. Shows no change over time
e. Don’t know
7. Personality changes with age:
a. Occur rapidly in advanced age
b. Are minimal and remain stable through
adulthood
c. Occur slowly but are marked when
comparing young and old
d. Occur early in mid-life and then remain
stable
e. Don’t know
B2. GERONTOLOGICAL PRACTICEBASED SOCIAL WORK QUIZ (NEW)
 12 items, multiple choice, Similar to ASWB Licensure Items
 Example
4. A social worker made a home visit to an older person and
assessed the client’s mental status as alert and oriented. Three
weeks later the social worker makes a follow-up visit and finds the
older person disoriented and incoherent. What should the social
worker assess for FIRST?
a. Depression
b. Dementia
c. Delirium
d. Alzheimer’s Disease
e. Don’t know
C. KNOWLEDGE OF SERVICES FOR
OLDER ADULTS
 12 items
 Matching of Community- based Services
 Social work higher than physicians and little change factor
Source: Damron-Rodriguez, J.A., Frank, J., Heck, E., Liu, D., Sragow, S., Cruise, P., & Osterweil, D.
(1998). Physician knowledge of community-based care: What’s the score? Annals of Long-Term
Care, 6(4), 112-121.
D. HPPAE GERIATRIC SOCIAL WORK
COMPETENCY SCALE II

Domains of geriatric social work skills relevant to field education

I. Values, ethics and theoretical perspectives

II. Assessment

III. Intervention

IV. Aging services, programs and policies

V. LEADERSHIP IN THE PRACTICE ENVIRONMENT OF AGING*

10 items in each domain (Total=50)

Rating on 5-point Likert scale
0 ----------------- 1 ------------------ 2 ------------------ 3 ---------------- 4
Not skilled at all - Beginning skill - Moderate skill - Advanced skill - Expert skill
*NEW
Leadership Competencies:
The Latest Additions
• Identified through HPPAE Consensus Panel
• Guided by Principles for Gero Education and
Practice (CalSWEC Aging Initiative)
• Focused on practice in the community and policy
arena
• Relevant for both micro/macro practice
• Considered life-long learning goals
Multiple Goals for
Competency Measurement
Most Improved Competency
Pre
Post
Diff
Pre-Post Test
Outcomes Pre-Post
M (SD) Min-Max
Variables
ST PRE
ST POST
Knowledge of Aging for Social
Work Quiz (0-25)
10.8 (2.9)
12.7 (2.7)
0-22
6-24
I. Values, ethics & theoretical
perspectives
16.7 (6.2)
27.6 (5.5)
0-33
10-40
12.2 (7.3)
25.7 (7.0)
0-37
2-40
12.0 (6.7)
24.8 (6.9)
0-30
4-40
8.6 (5.9)
21.2 (8.0)
0-28
0-40
49.5
(23.4)
99.3
(24.8)
3-115
30-160
(2001-2003
Understand common terminology used
in service provision
5.0
8.2
3.2
Creatively use organizational policy,
procedures & resources
4.6
7.7
3.1
Identify and incorporate roles of social
work and other profession into practice
5.3
Effectively use knowledge of program
outreach techniques
4.6
Incorporate full continuum of services
from home to institutional settings
4.6
8.4
3.1
II. Assessment
7.7
3.1
III. Intervention
7.5
2.9
IV. Aging services, programs, &
policies
GSWC Scale-II Total
*** p < .001
p
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
E. FIELD INSTRUCTOR VERSION
COMPETENCY SCALE II
 A measurement beyond self efficacy
 Administer of the Pre-test after acquaintance
period
 “Do not know” category added
Field Instructor
Rating of Student
Competence
Student SelfAssessment of
Geriatric Competencies
50
50
Pre-test
45
45
40
Post-test
40
35
35
30
30
25
25
20
20
15
15
10
10
5
5
0
0
Values
p<.001***
Assessment
p<.001***
Intervention Aging services
p<.05*
p<.05*
Values
p<.001***
Assessment
p<.001***
Intervention
p<.001***
Aging services
p<.001***
National Incorporation of Gero Social
Work Competencies
I. Council on Social Work Education (CSWE) 2008
Educational Policy and Accreditation Standards
(EPAS)

a competency-based outcome approach to
curriculum design

a focus on student outcomes based in
practice behaviors

competencies are measurable practice
behaviors
II. Hartford Geriatric Social Work Initiative
Task Force for Advanced Gero Practice (2009)
Relating Gero Social Work Knowledge and Practice
Behaviors to EPAS
Future Research Questions for
Social Work Education
Competency
Generalist/ Advanced
Knowledge, skills that all SW
students should have about
aging/ Knowledge, skills
required for specialized
practice in aging
Competency Level of
Social Work Education
(Norming)
 Baccalaureate
 MSW
 Post-graduate
 Relationship of GSW
Competencies to
EPAS
 Student background
variables/experience
and outcome
 Career variables and
competency/
knowledge outcomes
 Validity/reliability of
Field Instructor GSW
Scale
Noell Rowan
Associate Professor
University of North Carolina,
Wilmington
www.socialworkleadership.org
Publication
• Rowan, N. L., Faul, A. C., Birkenmaier, J.,
Damron-Rodriguez, J. (2011). Social work
knowledge of community-based services for
older adults: An educational model for social
work students. Journal of Gerontological
Social Work, 54 (2), 189-202.
www.socialworkleadership.org
Rationale for this article
• Initiated through discussions with JoAnn
Damron-Rodriguez about her prior work with
physicians and measuring knowledge about
community-based services for older adults.
• Desire to know more about social work
student knowledge
www.socialworkleadership.org
Measures utilized
• Pre-post KASW
• Geriatric Social Work competency
scale II
• Pre-post Knowledge of services for
older adults (specific relevant items)
www.socialworkleadership.org
Results
• Sample of N=353
• Students from 35 schools
• Demonstrates positive outcomes from
HPPAE model
• Statistically significant improvement from
pre-post test scores on KASW and GSW
Competency Scale II
• Positive trend indicated in pre-post
measures from Knowledge of Services for
older adults
www.socialworkleadership.org
Future Research Questions for
Social Work Education
• More work is needed to continue to
evaluate HPPAE model as to if education
improves scores on measures
• What efforts are being made to improve
social work knowledge and skills with
community-based services for older adults?
www.socialworkleadership.org
Publication
Greenfield, E., Morton, C., Birkenmaier, J., &
Rowan, N. L. (2013). Optimizing geriatric
social work education: Program individual
characteristics that promote competencies.
Journal of Gerontological Social Work, 56,
356-377.
www.socialworkleadership.org
Rationale for this article
• Initiated discussion among colleagues in
national conferences about next steps to
use HPPAE data
• Interest in using large dataset regarding
program and individual student
characteristics and ways to optimize
geriatric social work education
www.socialworkleadership.org
Theoretical Framework
• Bioecological systems theory provides
framework for exploring the extent to which
individual student and program
characteristics are associated with
students’ competence in aging
www.socialworkleadership.org
Results
• HLM was used to examine associations
between individual student and program
characteristics and changes in students’
competency in social work and aging.
www.socialworkleadership.org
Where were knowledge and
competency gains?
• Full time students showed greater gains in
knowledge on aging- most robust and largest
association between individual student
characteristics and changes in
competencies.
• Having a field rotation with more structure
(concurrent and sequential) demonstrated
increased knowledge and competency gains.
www.socialworkleadership.org
Where were knowledge and
competency gains?
• Advanced level students showed higher
knowledge gains as compared with
generalist level students
www.socialworkleadership.org
Future Research Questions for
Social Work Education
• Further research to examine student and
program characteristics to more fully
understand student knowledge gains about
aging services
www.socialworkleadership.org
Julie Birkenmaier,
Associate Professor
Saint Louis University
www.socialworkleadership.org
Social work knowledge of facts on aging: Influence of
field and classroom education
Methods:
N=323 student participants at 11 universities
Measure: Palmore Facts on Aging Quiz
Pre- and post-test data collected
Results:
-Improvement in knowledge scores (p=.0001) for graduates of the
enhanced field education programs.
-Taking prior aging course work was a significant (p=.0001)
influence on knowledge scores pre- and post-test.
Discussion:
-Increased knowledge of aging was found when experiential and
didactic learning was combined
Birkenmaier, J. M., Rowan, N., Damron-Rodriguez, J., Lawrance, F. & Volland, P (2009). Educational Gerontology,
35(9), 784-800.
www.socialworkleadership.org
Knowledge outcomes within rotational models of
social work field education
Methods: 231 students in the HPPAE project took the KASW preand post-test.
Results:
-Students in concurrent rotation model showed higher posttest
knowledge score
-The number of ways in which programs use geriatric social work
competencies positively influences posttest knowledge scores
-Students who completed field education in long-term care facilities
experienced higher posttest knowledge scores
Birkenmaier, J. M., Curley, J, & Rowan, N. (2012). Journal of Gerontological Social
Work, 55(4), 321-336.
www.socialworkleadership.org
Future Research Questions for
Social Work Education
-What is the optimal field and classroom content and structure needed
to maximize knowledge gains in combined experiences?
-What would an optimal curriculum for preparation in serving older
adults, one composed of both coursework and aging-enriched field
experience, look like?
-Why do students experience a higher level of knowledge in
concurrent field models?
-Do concurrent field education situations in which students maintain
one field instructor differ relative to knowledge outcomes from
concurrent field education in which more than one field instructor is
Involved?
-What is the optimal number, use, and mix of geriatric social work
competencies in field education?
www.socialworkleadership.org