DOCUMENTING EVIDENCE OF
EFFECTIVENESS IN YOUR
APPROVED PROVIDER UNIT
FALL, 2014 WEBINAR SERIES
Educational Design Process
1
Ohio Nurses Association and
Montana Nurses Association
(c) ONA, 2014
SERIES PLANNERS & FACULTY
Pam Dickerson, PhD, RN-BC, FAAN
Director of Continuing Education, Montana Nurses Association
Terry Pope, MS, RN
President, Nursing Institute for Continuing Education
Zandra Ohri, MA, MS, RN
Director of Continuing Education, Ohio Nurses Association
2
(c) ONA, 2014
SESSION 3
EDUCATIONAL DESIGN PROCESS: PROVIDING THE
EVIDENCE
Objective: Identify evidence to validate the
effectiveness of your provider unit in use of the
educational design process for your learning
activities.
3
(c) ONA, 2014
DISCLOSURES

Requirements for successful completion:
Attendance at event & turning in an evaluation
form.

Conflict of interest: Planners & faculty have
declared no COI

The Ohio Nurses Association (OBN-001-91) is
accredited as a provider of continuing nursing
education by the American Nurses
Credentialing Center’s Commission on
Accreditation.
(c) ONA, 2014
4
OF NOTE
 This
webinar is designed to increase your
effectiveness in documenting evidence of your
work related to the EDP criteria – it is NOT
intended to address the criteria themselves.

If you have specific questions about things like
how to do a needs assessment, assess for conflict
of interest, or negotiate a commercial support
agreement, please contact your nurse peer review
leader.
(c) ONA, 2014
5
6
(c) ONA 2014
OVERVIEW OF EDP CRITERIA DOMAINS
 EDP
1 & 2: Assessment
 EDP
3-6: Planning
 EDP
7-11: Design Principles
 EDP
12-13: Achievement of Objectives
7
(c) ONA, 2014
EDP 1 AND 2: ASSESSMENT OF LEARNING NEEDS
 Criterion:
CNE activities are developed in response to,
and with consideration for, the unique educational needs
of the target audience.

EDP 1 – Method of assessing learning needs of the target
audience

EDP 2 – Use of data to develop a learning activity to address
a gap in knowledge, skill, or practice
8
(c) ONA, 2014
EDP 3 – 6: PLANNING
 Criterion:
Planning for each educational activity must
include one nurse planner and one other planner. One of
the planners must have appropriate subject matter
expertise for the educational activity.




EDP 3 – Process to select a planning committee
EDP 4 – Process to identify a conflict of interest
EDP 5 – Process for resolution of a conflict of interest
EDP 6 – Process used to determine requirements for successful
completion
(c) ONA, 2014
9
EDP 7 - 11: DESIGN PRINCIPLES
 Criterion:
The educational design process incorporates
measurable educational objectives, best available
evidence, and appropriate teaching methods.





EDP 7 – Developing measurable objectives related to change in
practice or nursing professional development
EDP 8 – Selection of content based on best available evidence
EDP 9 – How content integrity is maintained
EDP 10 – Precautions with commercial support / sponsorship
EDP 11 – Selection of teaching methods to achieve purpose and
objectives
(c) ONA, 2014
10
EDP 12 – 13: ACHIEVEMENT OF OBJECTIVES
 Criterion:
A clearly defined method that includes learner
input is used to evaluate the effectiveness of each
educational activity. Results from the activity evaluation
are used to guide future activities.

EDP 12 – How summative evaluation data are used to guide
future activities

EDP 13 – How evaluation data demonstrated change in nursing
practice or nursing professional development
11
(c) ONA, 2014
WRITING TO THE CRITERIA
 First


2-3 paragraphs, or about ½ page
Clearly articulate your process, focusing on key words
 Then



– provide a description
– provide an example
One specific instance – the more detail, the better
Focus on how you implemented the process described
Keep internal consistency in mind – will your sample activities
support your description and example?
12
(c) ONA, 2014
EDP 1: ASSESSMENT OF LEARNING NEEDS
 Nurse
Planner’s methods of assessing the current
learning needs of the target audience
 Key
Points

Methods used by the activity’s nurse planner

Current learning needs

Target audience for a specific learning activity
13
(c) ONA, 2014
EDP 1: THINGS TO CONSIDER
 First:
What is the opportunity for improvement or the
problem in practice?
 Second:
 Third:
What outcome is desired to address this issue?
Who needs this information?
 Fourth:
Why do they need it? ** this is your needs
assessment for an activity!

Be sure you aren’t doing “wants” or “likes” assessments!
(c) ONA, 2014
14
EDP 2: ASSESSMENT OF LEARNING NEEDS
 How
the Nurse Planner uses data collected to develop an
educational activity that addresses the identified gap in
knowledge, skills, and/or practice.
 Key
points:

Use of data

Gap analysis

Development of activity
15
(c) ONA, 2014
EDP 2: THINGS TO CONSIDER
 What
 Why
is a “gap analysis”?
does it matter?
 When
planning an activity, how does this process form
the foundation for determining the desired outcome, then
planning the activity?
16
(c) ONA, 2014
EDP 3: PLANNING
 The
process used to select a planning committee for an
educational activity, including why an individual
member was chosen.
 Key
points:

Planning committee

Selection – process and a specific member
17
(c) ONA, 2014
EDP 3: THINGS TO CONSIDER
 Planning
committees should reflect relevance to the
activity being planned – be cautious of “generic” planning
committees
 Remember
internal congruence – if you state that
speakers are always on the planning committee,
reviewers will look for that in your sample activities
 What
is the role of the planning committee? Provide clear
evidence.
(c) ONA, 2014
18
EDP 4: PLANNING
 The
process used to identify all actual and potential
conflicts of interest for all members of the planning
committee, presenters, authors, and content reviewers.
 Key
points:

Use of a standard process

Knowledge of conflict of interest (COI)

Relevance for everyone who can control content
19
(c) ONA, 2014
EDP 4: THINGS TO CONSIDER
 Clearly
define COI – remember that it relates to people,
not organizations
 Focus
on identification of the COI – how do you do that?
 Don’t
digress into other topics
20
(c) ONA, 2014
QUESTION
 Is
“completion of the BIO/COI form” adequate to identify
conflict of interest?

Yes

No
21
(c) ONA, 2014
EDP 5: PLANNING
 The
process for resolution of an actual or potential
conflict of interest and the outcome achieved.
 Key



points:
Resolving a COI – what are the options?
What is achieved when you exercise those options?
NOTE: If you have never had to resolve a COI, answer this by
providing an “as if” example – not applicable is not acceptable!
22
(c) ONA, 2014
EDP 6: PLANNING
 The
process utilized during the planning phase of the
educational activity to determine how participants will
successfully complete the learning activity.
 Key
points:

Part of planning process

Criteria for successful completion – not disclosure, but decision
about what these will be

Why that decision was made
23
(c) ONA, 2014
EDP 7: DESIGN PRINCIPLES
 How
measurable educational objectives are developed
that address the change in nursing practice or
professional development.
 Key
points:

Measurable objectives
 What can be “measured”
 Avoid 2 or more verbs in one objective

Address change in practice or professional development
 Relevance to identified gap, purpose, and outcome measure
(c) ONA, 2014
24
EDP 7: THINGS TO CONSIDER






How do you develop objectives for your learning activities?
Who does this? If the presenter, how do the nurse planner and planning
committee have input?
How do you assure that the objectives are tied to the gap data and desired
outcome?
How do you assure that the objectives focus on the expectation for the
learner, not the presenter?
How are the objectives related to the teaching methods?
How do the objectives relate to change in nursing practice or professional
development?
(c) ONA, 2014
25
QUESTION ABOUT OBJECTIVES
 Is
it ok to give a blank planning table to a speaker to be
filled out and returned to the nurse planner?

Yes

No
26
(c) ONA, 2014
WORKING WITH THE PLANNING TABLE
 What’s
the risk of the speaker completing this form
independently?
 How
does the speaker know what’s been identified as the
practice gap or desired outcome?
 What
if you get data back that isn’t “right”?
27
(c) ONA, 2014
EDP 8: DESIGN PRINCIPLES
 How
the content of the educational activity is selected
based on best available current evidence
 Key
Points:
 Content selection
 Based on “best available evidence”
28
(c) ONA, 2014
EDP 8 - EXAMPLE
 The
speaker completes the planning table with
the objectives and content. The nurse planner
reviews the content to be sure it is on target with
the expected presentation and will help the
learner achieve the stated outcome. The speaker
also provides 3-5 references that are dated within
the past 5 years. If references are not provided, or
if they are out-of-date, the nurse planner will
contact the speaker to get better references.
(c) ONA, 2014
29
WHAT DO YOU THINK?
 Is
this a strong example to address EDP 8?

Yes

No
30
(c) ONA, 2014
THINGS TO THINK ABOUT
 Is
it always appropriate for the speaker to do this work
independently?
 What
about content in relation to objectives and teaching
methods?
 What’s
“magic” about 5 years?
 Just
because a reference is dated within the past 5 years,
does that mean it’s the “best available evidence”?
 What
are credible sources of evidence?
31
(c) ONA, 2014
EDP 9: DESIGN PRINCIPLES
 How
content integrity is maintained for CNE activities,
including what precautions are taken to prevent bias and
how those precautions are implemented
 Key



Points:
Content integrity
Bias
Implementing precautions
32
(c) ONA, 2014
WHAT IS “CONTENT INTEGRITY”?
 Actual
or potential conflicts of interest are resolved
 Content
is based on best on best available evidence
 Independence
from organization providing commercial
support or sponsorship
 Freedom
from promotional activity
33
(c) ONA, 2014
EDP 10: DESIGN PRINCIPLES
 In
the presence of commercial support/sponsorship, how
additional precautions are taken to maintain content
integrity for CNE activities, including what precautions
are taken to prevent bias and how those precautions are
implemented.



Definition of commercial support
Definition of sponsorship
Not applicable if you never have had and never expect to have
commercial support
(c) ONA, 2014
34
EDP 10: THINGS TO THINK ABOUT
 “Standard”
commercial support / sponsorship agreement
 What
else do you need to consider?
 What
precautions do you take:



Before an activity
During an activity
Note: If you have received commercial support or sponsorship
within the past 2 years, describe your process, provide an example,
AND attach an activity file with the commercial
support/sponsorship evidence.
(c) ONA, 2014
35
EDP 11: DESIGN PRINCIPLES
 How
teaching methods were selected that are
appropriate to achieve the purpose and objectives of the
CNE activity
 Key



Points:
Selection of teaching methods for an activity
Relationship of teaching methods to other components and
desired outcome
Relationship of teaching methods to criteria for successful
completion
36
(c) ONA, 2014
EDP 12: ACHIEVEMENT OF OBJECTIVES
 How
summative evaluation data for an educational
activity were used to guide future activities.
 Key
points:
 Summative evaluation
 Use of data
37
(c) ONA, 2014
EDP 12: SUMMATIVE EVALUATION
 Things
to consider:

What evaluation methods are you using?

Are the methods themselves giving you helpful data?

Who compiles evaluation data?

Who analyzes evaluation data?

Is there a “post-action” report or planning committee meeting to
analyze the outcome?

What do you do with the evidence from the summative
evaluation to guide future activities?
38
(c) ONA, 2014
EDP 13: ACHIEVEMENT OF OBJECTIVES
 How
evaluation data were collected to measure change in
nursing practice or nursing professional development

Key Points:
 Evaluation data use
 Measuring change
 Nursing practice? Professional development?
39
(c) ONA, 2014
EDP 13: MEASURING OUTCOMES
 Refer
to initial gap analysis and needs assessment data
 What
was the original “problem in practice” or
opportunity for improvement?
 How
do you know that issue has been addressed – or not?
40
(c) ONA, 2014
SUMMARY
 The
educational design process is the heart and soul of
your provider unit.
 Providing
evidence of adherence to criteria demonstrates
your commitment to quality continuing education to
enhance nursing practice and professional development.
 Coming
next: Quality Outcomes
41
(c) ONA, 2014
FOR FURTHER INFORMATION
 Pam




Dickerson, PhD, RN-BC, FAAN
Director, Continuing Education
Montana Nurses Association
Telephone: 406-465-9126
pam@mtnurses.org
 Zandra




Ohri, MA, MS, RN
Director, Continuing Education
Ohio Nurses Association
Telephone: 614-448-1027
zohri@ohnurses.org
42
(c) ONA, 2014