Documenting Evidence of Effectiveness in
Your Approved Provider Unit
Fall, 2014 Webinar Series
Quality Outcomes
Ohio Nurses Association and
Montana Nurses Association
(c) ONA, 2014
1
Series Planners & Faculty
Pam Dickerson, PhD, RN-BC, FAAN
Director of Continuing Education, Montana Nurses
Association
Terry Pope, MS, RN
President, Nursing Institute for Continuing Education
Zandra Ohri, MA, MS, RN
Director of Continuing Education, Ohio Nurses Association
(c) ONA, 2014
2
Session 4
Quality Outcomes: Providing the Evidence
 Objective: identify
evidence to validate the
effectiveness of your provider unit in collecting and
analyzing data reflective of achievement of goals and
quality outcome measures.
(c) ONA, 2014
3
Disclosures
◦ Requirements for successful completion: Attendance at
event & turning in an evaluation form.
◦ Conflict of interest: Planners & faculty have declared no
COI
◦ The Ohio Nurses Association (OBN-001-91) is
accredited as a provider of continuing nursing
education by the American Nurses Credentialing
Center’s Commission on Accreditation.
(c) ONA, 2014
4
(c) ONA, 2014
5
Overview of Quality Outcomes Section

Overall vision: The approved provider unit engages in an
ongoing evaluation process to analyze its overall
effectiveness in fulfilling its goals and operational
requirements to provide quality CNE.
What does this mean?
 What are you expected to do?

(c) ONA, 2014
6
Overview: Subsets

The approved provider unit (APU)must evaluate the effectiveness of
its overall functioning.

The APU shall include a variety of stakeholders, comprising those
with a vested interest in the provider unit’s outcomes, in the
evaluation process.

The APU must demonstrate quality improvement efforts, including
identifying strategies for working on targeted goals, evaluating
progress toward goals, and revising or establishing new goals.

The APU shall evaluate data to determine how the APU, through
the learning activities it has provided, has influenced the professional
development of its nurse learners.
(c) ONA, 2014
7
Key Questions

What is your evaluation process?

How does it contribute to changing quality outcome
measure(s)?

Who participates in the evaluation process, and how do they
help?

What have you done to improve the professional practice of
your RN learners?
(c) ONA, 2014
8
QO 1: Evaluation Process

The process utilized for evaluating effectiveness of the
APU in delivering quality CNE.

Key components:
◦
◦
◦
◦
Process
Effectiveness
Provider unit
Delivering quality education
(c) ONA, 2014
9
QO 1: Things to Think About

What do you include in your provider unit evaluation?

This is DIFFERENT THAN evaluating an individual
activity!

Consider all the factors that impact your ability to
provide quality CNE: human, material, and financial
resources; time; money; alignment with the strategic plan
of the organization; departmental goals; metrics
associated with quality outcome measures
(c) ONA, 2014
10
QO 2: Evaluation Process

How the evaluation process for the APU resulted in the
development or improvement of an identified quality
outcome measure (refer to QOMs listed in OO4).

Key components:
◦ Evaluation process identified in QO 1
◦ Developing and/or improving quality outcome measures (really
important to tie these pieces together)
(c) ONA, 2014
11
QO 2: Things to Think About

What is your process for making changes in the
structure and operations of your provider unit? What
evaluation data supports those changes?

Are your quality outcome measures in OO4 appropriate,
realistic, and measurable?

What is an example of how you’ve developed a new
QOM or strengthened an existing one?
(c) ONA, 2014
12
Making QOM Data Clear
Which outcome measure is more clearly written?
1.
Learner engagement.
2. At least 75% of activities will involve active
engagement of learners.
(c) ONA, 2014
13
Relating QOMs to Evaluation Data
Which set of provider unit quality outcome measures
would be the appropriate “home” for the QOM about
learner engagement?
1.
OO4b:Your provider unit
2.
OO4c: Nursing professional development
(c) ONA, 2014
14
If That Was Your QOM…

75% of activities in calendar year 20xx will involve active
engagement of learners.

What is the process for evaluating the provider unit in
relation to activities with learner engagement? Were you
successful? If not, why not? Were faculty not aware of how to
do this? Were learners unwilling? Were activities not
structured appropriately? Were physical resources not
conducive?

So what – what did you do? Provide a specific example of
how you improved this QOM based on your evaluation data.
(c) ONA, 2014
15
QO 3: Evaluation Participants

Why the APU selects specific stakeholders to participate
in the evaluation process.

Key components
◦ Selection – an active decision on your part, not just “by default”
◦ Specific people or groups (education council, for example)
◦ Active participation – not just names on a form
(c) ONA, 2014
16
QO 3: Things To Think About

What are stakeholders?

Why do they matter?

How do you decide who participates?

Are they clear on the importance of their roles?

What, specifically, do you want them to contribute to the
evaluation process?
(c) ONA, 2014
17
What About This?
Is this an ok response to identification of stakeholders?

They are our nurse planners and faculty
◦ Yes
◦ No
(c) ONA, 2014
18
QO 4: Quality Outcome Measures

How input from stakeholders resulted in development or
improvement in quality outcome measures for the APU.
(Again, refer to those QOMs in OO4b)

Key components:
◦ Input from stakeholders (they aren’t just names on the list in
QO3)
◦ Use of their feedback
◦ Correlation to QOMs
(c) ONA, 2014
19
QO 4: Things To Think About

Same question as with QO3: do stakeholders really appreciate the
value of their roles? Do they understand the importance of what
they do?

Are you engaging the right stakeholders? Are there opportunities
for expanding your pool (or contracting it)?

Are they well-versed in your current QOMs to be able to provide
input?

Are you giving them data that you’ve collected to help inform their
comments and suggestions?
(c) ONA, 2014
20
QO 5:Value/Benefit to Nursing Professional
Development

How, over the past 12 months, the APU has enhanced
nursing professional development (again, referring to the
QOMs in OO4).

Key components:
◦ 12 months (not the whole provider approval period)
◦ Enhancing nursing professional development – in the aggregate
◦ Related to QOMs in 004c
(c) ONA, 2014
21
QO 5: Things to Think About

Reflect on your goals and outcome measures

Why were these important? Were they stated in measurable terms?
If not, how do you know if you’ve made progress?
◦ Improve responsiveness to patients whose conditions are deterioriating
◦ Decrease response time for rapid response teams on med-surg units by
50%

What do we mean by “nursing professional development”?
◦ May not be a direct patient-care change
◦ Requires strong knowledge of your target population and their needs
(globally, not just for a particular learning activity)
(c) ONA, 2014
22
QO 5: The Bottom Line

Are you worth keeping around?

Does your organization need you?

Can you validate your contributions to the strategic plan
of your organization?

This is your evidence – make it strong!
(c) ONA, 2014
23
Summary
This concludes our 4-part series on “Documenting the
Evidence”
Organizational Overview – general information about
your APU and your organization
 Structural Capacity
 Educational Design Process
Description and Example
 Quality Outcomes

(c) ONA, 2014
24
Self-Assessment
I am able to:
1 – Not at All
2 - Somewhat
3 – Fairly Well
4 –Yes!
Write goals and
quality outcome
measures for my
APU
Determine what
should be included
in evaluating my
APU
Select stakeholders
to participate in our
evaluation process
Provide evidence of
our success
Your Score:
(c) ONA, 2014
25
Self-Assessment

My score is between:
◦
◦
◦
◦
4-7
8-10
11-13
14-16
(c) ONA, 2014
26
Questions and Comments
(c) ONA, 2014
27
For Further Information

Pam Dickerson, PhD, RN-BC, FAAN

Zandra Ohri, MA, MS, RN
◦
◦
◦
◦
◦
◦
◦
◦
Director, Continuing Education
Montana Nurses Association
Telephone: 406-465-9126
pam@mtnurses.org
Director, Continuing Education
Ohio Nurses Association
Telephone: 614-448-1027
zohri@ohnurses.org
(c) ONA, 2014
28