www.ualberta.ca/~kusp
The Alberta Context Tool (ACT)
Carole A. Estabrooks
Professor &
Canada Research Chair
Janet Squires
Doctoral candidate
Faculty of Nursing
University of Alberta
KU09
Wales, UK
June 24-26, 2009
k n o w l e d g e i n p r a c t i c e. . .
www.ualberta.ca/~kusp
ACT overview
•
59 items representing 8 modifiable dimensions of
context
•
Currently
3 versions
• Adults (acute care)
• Pediatrics (acute care)
• Long term care
6 groups
•
•
•
•
•
•
Healthcare Aides (LTC only)
Nurses
Allied
Physicians
Specialists/Educators
Managers
k n o w l e d g e i n p r a c t i c e. . .
www.ualberta.ca/~kusp
Initial ACT development
1. Selection of a conceptual framework
•
•
Critical review of literature
Promoting Action on Research Implementation in
Health Services (PARiHS) framework as initial guide
2. Conceptual refinement
•
•
Addition of concepts to incorporate team’s
understanding of other relevant aspects of
organizational context
Conceptual and operational definitions developed
k n o w l e d g e i n p r a c t i c e. . .
www.ualberta.ca/~kusp
Initial development
3. Survey construction
•
•
Item construction
Initially versions for 5 groups in adult acute care (RN/LPN,
MD, Allied, Managers, Educators)
4. Feasibility modifications
•
•
Survey administration and debriefing sessions were held with
members of the five groups (summer, 2006)
Modifications made based on debriefing feedback
•
•
•
Question clarity
Revision of examples
Length/respondent burden
5. Response process validity evidence
k n o w l e d g e i n p r a c t i c e. . .
Framework
PARiHS Framework
Research implementation
Leadership
Adding to
context
Context
Culture
Evidence
Facilitation
Evaluation
Individual
attributes
Kitson, et al., 1998, QSHC
Rycroft-Malone et al, 2002 (J Adv Nurs)
+
• Slack
• Interactions
• Resources
• Social capital
• Attitude
• PSA
• Belief suspension
• MBI
• SF-8
ACT Framework
Leadership
Adding to context
Context
Culture
Evaluation
• Organizational slack
• Formal interactions
• Informal interactions
• Structural &
electronic resources
• Social capital
Structure of ACT
1. Leadership
• 6 items scored on a 5-point Likert like scale
• Concepts reflected in items: openness, optimism, self control, em
pathic, developing others, conflict management
• e.g., The leader calmly handles stressful situations
2. Culture
• 6 items scored on a 5-point Likert like scale
• Concepts reflected in items: recognition, autonomy, worklife balan
ce, development opportunity, focus on service/mission, support
• e.g., I receive recognition from others about my work
3. Evaluation
• 6 items scored on a 5-point Likert like scale
• Concepts reflected in items: data access, informal data review, for
mal data review, action planning, performance monitoring, bench
marking
• e.g., Our team routinely discusses this data informally
Structure of ACT
4. Formal Interactions
• 5 items scored on a 5-point frequency scale
• Concept reflected in items: Interactions with others
through engagement in formal organizational (unit) activities
• e.g., Team meetings about residents
5. Informal Interactions
• 6-9 items scored on a 5-point frequency scale
• Concept reflected in items: Interactions with others
through engagement in informal organizational (unit)
activities
• e.g., Other professionals in my discipline
6. Social Capital
• 6 items scored on a 5-point Likert like scale
• Concepts reflected in items: bonding, bridging, linking
• e.g., People in my group share information with others in the
group
Structure of ACT
7. Structural and Electronic Resources
• Structural: 9 items scored on a 5-point frequency scale
• Electronic: 3 items scored on a 5-point frequency scale
• Concept reflected in items: availability/use of structural and
electronic resources
• e.g., Policy and procedure manuals
8. Organizational Slack
• 9-11 items consisting of 3 sub-scales
• Staffing (2-3 items scored on a 5-point Likert like scale
assessing availability of adequate staffing resources
• Space (3-4 items scored on Likert and frequency scales
assessing availability and use of space
• Time (4 items scored on a 5-point frequency scale
assessing availability and use of time)
• e.g., Have time to do something extra for residents
www.ualberta.ca/~kusp
Validation
k n o w l e d g e i n p r a c t i c e. . .
The Standards1 approach to validity
A unitary approach to validity assessment where empirical results
are classified as supporting (or refuting) validity
Validity Source
Explanation
Content
To what extent does the content of the items represent the
content of the domain? e.g., expert panel, CVI
Response
Process
To what extent do the respondents’ responses fit the
intended construct? Do respondents interpret, process, and
elaborate upon item content and is this behaviour in
accordance with the construct?
e.g., revise items from pretest, interview to confirm
instrument
Internal
Structure
To what extent do the relationships among the items match
the construct as operationally defined and do the data
support dimensionality?
e.g., factor analysis, item analysis (IRT)
Relationships
with other
Variables
What is the nature and extent of the relationships between
instrument scores and:
- variables it is expected to correlate with or predict
- variables it is not expected to correlate with or predict
e.g., correlations, regression
1
Standards for Educational and Psychological Testing. AERA; 1999.
www.ualberta.ca/~kusp
Initial validation
(AKUTE1 study)
• Acute Care (Adult) setting
• 4 teaching hospitals in Alberta
• 5 professional provider groups (N=453)
• Reliability (alpha)
• Range: .65 (formal interactions) to .92 (evaluation)
• All but one >.70
• Validity (internal structure evidence)
• Factor analysis (PCA with Varimax rotation)
 14 factor solution; 67% of the variance
 ACT core (culture, leadership, evaluation – 31.95% of variance)
1`AKUTE:
Estabrooks, Norton, Birdsell, Cummings, Newton. AHFMR funded
k n o w l e d g e i n p r a c t i c e. . .
www.ualberta.ca/~kusp
Second validation study
(pilot + wave 1 CIHR Team in Children’s Pain)
• Survey revisions made based on pilot findings from
four pediatric acute care units in Alberta (response
process)
• Validation study - acute care (pediatric) setting
• 8 pediatric hospitals across Canada
• 5 professional provider groups (N=1248)
• Validation with Nurses (N=850)
• Reliability (alpha)
• Range: .60 (formal interactions) to .91 (evaluation)
• All >.70 except for formal interactions (.60) and space (.65)
k n o w l e d g e i n p r a c t i c e. . .
www.ualberta.ca/~kusp
Second validation study
• Validity (internal structure evidence)
• Item assessment
• Item-total statistics remained stable
• Factor analysis
• 14 factor solution; 61% of the variance
• ACT core (culture, leadership, evaluation – 27.63%)
• Some realignment of items (formal/informal interactions)
k n o w l e d g e i n p r a c t i c e. . .
www.ualberta.ca/~kusp
Second validation study
• Validity (relationship with other variables evidence)
• Correlation analysis
• All ACT dimensions significantly correlated with IRU
• Range: r=.052 (OS-staff) to .272 (structural/electronic
resources)
• Regression Analysis
• 7/8 ACT dimensions predict research use (at hospital level)
at p<.10 (exception: structural/electronic resources)
k n o w l e d g e i n p r a c t i c e. . .
www.ualberta.ca/~kusp
Additional Validation
(CapitalCare studuy)
Pilot Study
•
91 LPNs and HCAs in 4 long-term care units
•
Extensive feasibility testing (response process validity
evidence)
•
Reliability:
• Alpha > .70 for 7/8 dimensions (formal interactions
revised)
• Alpha range 0.50 (formal interactions) to 0.96
(evaluation)
k n o w l e d g e i n p r a c t i c e. . .
www.ualberta.ca/~kusp
Ongoing plans
 The ACT currently under revision
• i.e., harmonize across versions , reduce the number of versions
 Planned validation:
• Psychometric assessment in new populations and in settings:
 Further psychometric assessment:
• Item Response Theory (IRT)
• Confirmatory Factor Analysis (CFA)
 Ongoing work re scoring of dimensions
 Assessment of translated versions (e.g., Swedish, French)
k n o w l e d g e i n p r a c t i c e. . .
www.ualberta.ca/~kusp
k n o w l e d g e i n p r a c t i c e. . .