April 28 2014
State of the Art Nursing Conference
Angie Andersen DNP, ACNP-BC
Angela Andersen has no financial interest or
arrangement that would be considered a
conflict of interest.

Angela Andersen, DNP, ACNP-BC, Nurse Practitioner, Palliative Care
Department, The Nebraska Medical Center, Omaha, NE.

Mary Parsons, PhD, RN, Associate Professor and Chair DNP Program,
Creighton University School of Nursing, Omaha, NE.

Regina Nailon PhD, RN, Clinical Nurse Researcher, The Nebraska Medical
Center Omaha, NE.

Sue Ann Gaster BSN, RN, Staff Nurse, Adult Progressive Care Unit, The
Nebraska Medical Center, Omaha, NE.

Rachael Mooberry BSN, RN, Staff Nurse, Oncology-Hematology
Specialty Care Unit, The Nebraska Medical Center, Omaha, NE.

Jane Meza, PhD, Professor College of Public Health Biostatistics,
University of Nebraska Medical Center, Omaha, NE.

Improve quality of life for patients and their
families facing the problems associated with
serious or life-threatening illness, through the
prevention and relief of suffering
World Health Organization , 2011

Palliative care teams utilize an interdisciplinary
approach in which physicians, nurses,
chaplains, social workers, and other allied
health professionals provide care
Weissman & Meier, 2011
Treat pain and other symptoms that
can cause complications in hospitalized
patients
 Establish goals for care
 Support family members in crisis
 Plan for safe transitions from hospital
to other settings

Weissman & Meier, 2011

Three levels of palliative care:
 Primary
▪ Basic skills and competencies required to manage the day-today patient care
 Secondary
▪ Treating physician refers to a specialist-level palliative care
provider for management of complex or difficult problems
 Tertiary
▪ Education and research
Von Gunten & Lupu, 2004; Weissman & Meier, 2011




Nearly half of all Americans die in a hospital
7 out of 10 Americans say they would prefer
to die at home
Only 25 % of Americans actually die at home
More than 80% of patients with chronic
disease say they want to avoid hospitalization
and intensive care when they are dying
Centers for Disease Control, 2005; Dartmouth Atlas of Health Care, 2005

Researchers examined medical records for
840,000 people 66 or older who died in 2000,
2005, and 2009:
 Increase use of hospice program in 2009, but more
than a quarter of hospice use was for 3 days or less, and
 40% of those late referrals followed a hospitalization
with an intensive-care stay
 Patients receive aggressive care until time of death and
did not receive full benefit of hospice care or program
Teno et al., 2013

Extensive body of evidence demonstrates
difficulties in providing adequate pain and
symptom management, as well as inconsistent
communication for hospitalized patients with
serious or life threatening conditions
Tilden et al., 1995; Hanson et al., 1997; Claessens et al., 2000; Lynn et al., 2000;
Norton & Talerico, 2000; Norton et al., 2002

The aggressive care provided in hospitals during
the last year of life accounts for approximately
12% of the U.S. health care budget and 27% of
Medicare expenditures
Centers Disease Control, 2009

Left unchecked, it is projected that health care
spending will increase 25% by 2030, largely
because of the aging population and chronic
disease
Centers Disease Control, 2009

Evidence suggests patients and families with
serious or life-threatening illness who
received palliative care interventions along
with standard care reported:




Improved physical and psychological symptoms
Improved quality of life
Longer median survival time
Improved family caregiver well-being
Lautrette,2007 ; Wright et al., 2008; Bakitas et al., 2009; & Temel et al., 2010

Despite the evidence, transition from diseasedirected treatment to an emphasis on palliative
care often occurs within days of end of life, if at all
Hui et al., 2010; Reville et al., 2010; Hi et al., 2012

Early identification of palliative care needs is
critical for clinicians to provide appropriate and
timely interventions directed at the specific level
of palliative care required by the patient and their
family
Weissman & Meier, 2011


General lack of assessment techniques that
would equip providers to identify palliative
care needs in hospitalized patients
Although a variety of palliative care
assessment instruments have been
developed, non have examined reliability or
validity to date
Bradley & Brasel, 2009; Fins, Miller et al., 1999;
Fins, Schwager et al., 2000; Imhof, Kaskie, & Wyatt, 2007

Investigating the psychometric properties of
an instrument is a common standard prior to
implementing the tool in clinical practice
http://www.jointcommission.org/accreditation/hospitals.aspx

The Centers for Medicare & Medicaid Services
and Joint commission on Accreditation of
Healthcare Organizations support the use of
evidence-based instruments and practices by
clinicians caring for hospitalized patients
http://www.jointcommission.org/accreditation/hospitals.aspx

The Center to Advance Palliative Care
(CAPC) developed a set of criteria to
identify patients at hospital admission
that would be appropriate to receive
further palliative care assessment and
interventions
Weissman & Meier, 2011
No study has examined the reliability
and other psychometric properties of
the CAPC criteria
 Reliability is a prerequisite for any kind
of validity, and is the degree to which
measurement error is absent from data

Polit & Beck, 2008

Interrater reliability is a specific type of
reliability referring to the amount of
agreement between different raters.
 In the case of the CAPC criteria, interrater
reliability is useful to measure whether two (or
more) raters independently come to an exact or
nearly exact agreement when scoring a patient
Polit & Beck, 2008

Although exact agreement of independent raters is
ideal, a small difference in rating is of minor clinical
relevance
Polit & Beck, 2012

Conversely, if the difference in assessment and
scoring between raters increases, it is very likely this
will have implications for clinical practice
Polit & Beck, 2012
 For example, whereas one nurse considers a patient at risk for
having unmet palliative care needs and provides intervention to
address the specific level of palliative care required by patient
and their family, another nurse may regard the patient not at
risk and will not consider any further interventions
The purposes of this pilot study were twofold:
 To establish the interrater reliability of CAPC criteria that
identify and trigger primary palliative care assessment at
hospital admission in adult patients admitted to oncology
and progressive care units who received palliative care
services during their hospitalization; and
 To describe the CAPC criteria identified most frequently in
study patients who met CAPC criteria at hospital admission

Design
 Retrospective, descriptive, exploratory
 Setting and Sample
▪ Midwestern academic medical center
▪ Randomized, purposive sample
▪ Using a sample of patients known to have received palliative care services
strengthened the study design and enhanced the study team’s ability to
determine the interrater reliability of the CAPC criteria
▪ A sample size of 100 was adequate to determine the instrument’s reliability
using the kappa statistic, along with a 95% confidence interval.
▪ Inclusion Criteria
▪ 19 years or older
▪ Admitted to the adult oncology or progressive care units between January 1
and December 31, 2011 who received palliative care services during their
hospitalization


The CAPC criteria comprise an instrument for
use at hospital admission to identify patients
whose conditions warrant a primary palliative
care assessment
National consensus panel developed criteria
from research findings, national standards,
and expert opinion
Weissman & Meier, 2011


The CAPC criteria has primary and secondary
criteria to facilitate identification of patients
appropriate for primary palliative care
assessment
The starting point for assessing any given
patient using the primary and secondary
criteria is the identification of patients with
potentially life-limiting or life-threatening
conditions
Weissman & Meier, 2011

Primary Criteria
 Includes the 5 most important indicators
identified by the consensus panel
 These criteria are global indicators that represent
the minimum that nurses and clinicians should use
to screen patients at risk for unmet palliative care
needs at hospital admission
Weissman & Meier, 2011
Potentially life-limiting or life-threatening condition
Surprise Question: You would not be surprised if the patient died within 12 months.
Frequent admissions (more than one admission for same condition within 3 months).
Admission prompted by difficult-to-control physical or psychological symptoms (e.g.,
moderate-to-severe symptom intensity for more than 24 hours).
Complex care requirements:
• Functional dependency
• Complex home support for ventilator
• Complex home support for antibiotics
• Complex home support feedings
In last 3 months, decline in:
• Function (mobility or mental capacity)
• Feeding intolerance (nausea, vomiting, or bloating)
• Unintended decline in weight (e.g., failure-to-thrive)

Secondary Criteria
 More specific indicators of higher likelihood of
unmet palliative care needs and are designed to
be used as supplemental criteria in hospitals with
more comprehensive palliative care services
available
Admission from long-term care facility or medical foster home
Cognitively impaired elderly (> 70 years) patient with acute hip fracture
Metastatic or locally advanced incurable cancer
Chronic home oxygen use
Out-of-hospital cardiac arrest
Current or past hospice program enrollee
Limited social support
No history of completing an advanced care planning discussion

Nurse investigators independently
reviewed the medical record of each
patient for evidence of CAPC criteria
present within 48 hours of patients
hospital admission
 The nurse investigators determined the presence of
one or more CAPC criteria that identified the need
for primary palliative care assessment
 The principal investigator reviewed all 100 medical
records and each co-investigator reviewed 50
Inter-rater reliability was examined with
the kappa statistic, along with a 95%
confidence interval
 A test for whether kappa is different
from zero was also calculated

 If the p-value < 0.05, we concluded that the
kappa value was significantly different from zero
Primary Criteria
N= 50
Life-limiting/threatening condition
Surprise question
Frequent admissions
Admit difficult-to-control symptoms
Functional dependency
Complex home support ventilator
Complex home support antibiotics
Complex home support feedings
Last 3 months, decline in function
Last 3 months, feeding intolerance
Last 3 months, decline in weight
1Unable
To Determine
1UTD
Yes
Count (%)
50 (100%)
No
Count (%)
0
Kappa
Value
p-value
Count (%)
0
1.000
-
2 (4%)
48 (96%)
0
1.000
-
35 (70%)
15 (30%)
0
0.854
< .0001
46 (92%)
4 (8%)
0
0.648
< .0001
22 (44%)
23 (23%)
5 (10%)
0.825
< .0001
0
50 (100%)
0
1.000
-
0
50 (100%)
0
1.000
-
6 (12%)
44 (88%)
0
1.000
< .0001
43 (86%)
5 (20%)
2 (4%)
0.742
< .0001
28 (56%)
18 (36%)
4 (8%)
0.817
< .0001
16 (23%)
15 (30%)
19 (38%)
0.757
< .0001
Kappa Value /Agreement:
1.000 = perfect
0.99 -0.81 = almost perfect
0.80-0.61 = substantial
0.60 or less = poor
Secondary Criteria
N= 50
p-value
Count (%)
43 (86%)
0
0.912
< .0001
0
50 (100%)
0
1.000
-
50 (100%)
0
0
1.000
-
8 (16%)
42 (84%)
0
0.702
< .0001
Out-of-hospital cardiac arrest
0
50 (100%)
0
1.000
-
Hospice program
0
50 (100%)
0
1.000
-
Limited social support
12 (24%)
38 (76%)
0
0.390
< .0025
No history advance care planning
15 (30%)
70 (53%)
0
0.595
< .0001
Cognitively impaired elderly hip fx.
Metastatic or incurable cancer
Chronic home oxygen
1Unable
To Determine
No
Count (%)
7 (14%)
1UTD
Kappa
Value
Admission long-term care facility
Yes
Count (%)
Kappa Value /Agreement:
1.000 = perfect
0.99 -0.81 = almost perfect
0.80-0.61 = substantial
0.60 or less = poor
Primary Criteria
N= 37
Life-limiting/threatening condition
1UTD
Yes
No
Count (%) Count (%) Count (%)
Kappa
Value
p-value
37 (100%)
0
0
1.000
-
0
36 (97%)
1 (3%)
1.000
-
Frequent admissions
9 (24%)
25 (68%)
3 (8%)
0.703
< .0001
Admit difficult-to-control symptoms
36 (97%)
1 (3%)
0
1.000
-
Functional dependency
24 (65%)
10 (27%)
3 (8%)
0.311
0.0067
Complex home support for ventilator
0
50 (100%)
0
1.000
-
Complex home support antibiotics
0
50 (100%)
0
1.000
-
3 (8%)
33 (89%)
1 (3%)
0.844
< .0001
Last 3 months, decline in function
24 (65%)
8 (22%)
5 (13%)
0.392
< .0001
Last 3 months, feeding intolerance
13 (35%)
21 (57%)
3 (8%)
0.712
< .0001
Last 3 months, decline in weight
9 (24%)
21 (57%)
7 (19%)
0.479
< .0001
Surprise question
Complex home support feedings
1
Unable To Determine
Kappa Value /Agreement:
1.000 = perfect
0.99 -0.81 = almost perfect
0.80-0.61 = substantial
0.60 or less = poor
Secondary Criteria
N= 37
Yes
Count (%)
No
Count (%)
Admission long-term care facility
9 (24%)
Cognitively impaired elderly hip fx.
1UTD
p-value
Count (%)
Kappa
Value
28 (76%)
0
0.924
< .0001
1 (3%)
36 (97%)
0
1.000
< .0001
Metastatic or incurable cancer
10 (28%)
26 (72%)
0
0.933
< .0001
Chronic home oxygen
9 (24%)
28 (76%)
0
0.853
< .0001
Out-of-hospital cardiac arrest
1 (3%)
36 (97%)
0
0.654
< .0001
0
37 (100%)
0
1.000
-
Limited social support
10 (27%)
17 (46%)
10 (27%)
0.510
< .0001
No history advance care planning
19 (51%)
18 (49%)
0
0.837
< .0001
Hospice program
1Unable
To Determine
Kappa Value /Agreement:
1.000 = perfect
0.99 -0.81 = almost perfect
0.80-0.61 = substantial
0.60 or less = poor
 Study sample revealed most
frequently identified CAPC
Criteria:
▪ Life-limiting condition
▪ Surprise question


Nurse investigators had perfect to substantial
agreement for the majority of the CAPC
criteria
Perfect to substantial agreement provides
confidence in nurses’ abilities to administer
and score the CAPC instrument for the study
population

Prior to making inferences about
interrater reliability of CAPC criteria,
it is important to note the limitations
of the CAPC instrument
 Poor level of agreement for four criteria:
▪ Limited social support
▪ Functional dependency
▪ In the last 3 months decline in function
▪ In the last 3 months decline in weight
 Lack of operational definitions

In clinical practice, it is common that a team
of interdisciplinary clinicians provide care to
patients and their families
 Nurses on the team have an essential role in
identifying unmet needs of patients and
coordinating services
 Therefore it is essential that any instrument used
to assess patients for palliative care needs has
findings that are repeatable between nurses

Retrospective study design may have
contributed to the nurse investigators’
inability to determine the presence or absence
of each criterion
 Quality of documentation
 Investigators knowledge and experience

Generalizability of study findings:
 Patients who received palliative care
 Definition of “hospital admission”


Identification of palliative care needs is
necessary for nurses and other clinicians to be
able to provide interventions directed at the
specific level of palliative care required by the
patient and their family
Establishing interrater reliability of the CAPC
criteria is a necessary first step in determining
the utility of having registered nurses
conduct the screening at hospital admission
Beginning of a process that will contribute to the
availability of data that describe the characteristics of
hospitalized patients appropriate for further palliative
care assessment and intervention
 Future examination of the interrater reliability of
CAPC criteria:

 Other patient populations
 Formal education of nurse investigators to increase
understanding of what each criterion is intended to
measure
 Describe clinically relevant differences between nurses’
disagreements


Nurses play an essential role in identifying
hospitalized patients who are at risk for
having unmet palliative care needs
Establishing interrater reliability of the CAPC
criteria is essential to determining the utility
of having RN’s conduct the screening at
hospital admission of patients who are at risk
for unmet palliative care needs


Based on the study findings, it is realistic and
suitable for nurses to administer and score
the CAPC criteria at hospital admission
Implementation of an established instrument
will provide the structure and process needed
to ensure consistent and timely identification
of patients at risk for having unmet palliative
care needs

This study is a first attempt at establishing
psychometric properties of the CAPC criteria
to identify and trigger further palliative care
assessment at hospital admission
Questions?