Scoring Examples, Provider Units, Nurse Peer Reviewer Evaluation
8.29.12
Provider Units
Accountability. The Primary Nurse Planner is accountable for ensuring that all Nurse Planners and key personnel
in the Provider Unit adhere to the ANCC accreditation criteria.
Describe and, using an example, demonstrate:
SC3. How the Primary Nurse Planner ensures that all Nurse Planners and key personnel of the Approved
Provider Unit maintain adherence to the ANCC accreditation criteria.
SAMPLE RESPONSE 1:
In our organization, the Primary Nurse Planner provides all Nurse Planners with a copy of the Approved Provider
manual and shows them how to access information from the Approver Unit web site. Each Nurse Planner is
responsible for finding the information she needs to know in order to function as a Nurse Planner. In
addition, the Primary Nurse Planner discusses Approver Unit requirements during the monthly staff
meetings.
Ann Smith, our newest Nurse Planner was given a copy of the Approved Provider manual to read by our Primary
Nurse Planner. The PNP also reviewed how to access the Approver Unit website. Ann was given time to
review the website.


SCORE: 2
Rationale:
o Narrative partially addresses criteria requirement
 Fails to demonstrate how the PNP ensures Nurse Planners and key personnel maintain
adherence to accreditation criteria
 Providing a copy of the manual and telling Nurse Planners where to find information fails to
demonstrate accountability to maintain adherence
 Fails to address key personnel
o Supporting evidence is lacking
 Insufficient description to demonstrate how PNP is accountable for ensuring all Nurse
Planners and key personnel maintain adherence to accredit
SAMPLE RESPONSE 2:
In our organization, the Primary Nurse Planner conducts an orientation for all Nurse Planners hired into the
organization. The PNP uses a standardized power point presentation for each orientation session and
provides the power point in hard copy for each Nurse Planner. Each new Nurse Planner is given a current
Approved Provider manual also. During monthly staff meetings, the PNP routinely asks if any Nurse
Planners have questions that need to be discussed. If any Nurse Planners have questions, the PNP
discusses them with the group. This is a great learning opportunity for all members of the Approved
Provider unit.
One example of a time that a question was raised about the planning process happened when Sally Johnson,
Nurse Planner, was unsure how to document a needs assessment for the educational activity. The PNP
used the planning document to show how a needs assessment was documented. Everyone agreed that
they learned from the presentation.


SCORE: 2
Rationale:
o Narrative partially addressed criteria requirement
 Does not address key personnel
o Supporting evidence is lacking
 Brief description of adherence but lacks sufficient detail to demonstrate accountability of
the PNP for maintaining adherence to accreditation criteria
SAMPLE RESPONSE 3:
1
Scoring Examples, Provider Units, Nurse Peer Reviewer Evaluation
8.29.12
In our organization, the Primary Nurse Planner provides an orientation for all new Nurse Planners and any new
members of the organization that will provide support for planning, implementing and evaluating educational
activities. The orientation program is standardized to ensure that it is comprehensive and all key elements
required for orientation are included each time. This is helpful in the event that the Primary Nurse Planner is
unavailable to provide the orientation (due to scheduling issues, for example) or when Nurse Planners and
key members of the organization want to review the orientation program at a later date. Each Nurse Planner
and key member is given a copy of the Approved Provider manual and also is shown how to access
information on the Approver Unit web site. Each Nurse Planner and key member is also given an orientation
booklet with sample documents and FAQs to help them learn the processes used by the organization. For
the first 6 months after hire, a Nurse Planner or key member is paired with an experienced Nurse Planner
who acts as a mentor in case the new Nurse Planner or key member has any questions or concerns. The
PNP routinely reviews the documentation and processes of the Nurse Planner or key member to identify any
issues that need to be addressed.
An example of the orientation program provided to new Nurse Planners and key members of the organization can
be found in Attachment 12.
One example of how the PNP identified and addressed a concern with a new Nurse Planner was during the
process of reviewing documentation in the first 6 months. The PNP identified that Sally Johnson, Nurse
Planner, routinely documented a needs assessment using the check boxes on the planning document, but
she failed to include any supporting evidence in the activity file. The PNP reviewed the documentation
process with Sally and showed her how to include supplemental evidence such as references in the
literature or a copy of the needs assessment findings from the annual organizational survey. The PNP then
reviewed Sally’s documentation for the next educational activity that Sally planned and determined that the
documentation process was much improved and that supplemental evidence was in the file.


SCORE: 3
Rationale:
o Narrative answers the question and responds to criteria requirement
 The narrative discussed the process for assuring adherence
o Supporting evidence demonstrates that the criteria requirement is met
2
Scoring Examples, Provider Units, Nurse Peer Reviewer Evaluation
8.29.12
SC4. How the Primary Nurse Planner is accountable for resolving issues related to providing CNE.
SAMPLE RESPONSE 1:
As the Primary Nurse Planner, my position in the organization is recognized as the individual accountable for
ensuring that all educational activities awarding CNE credit are planned, implemented and evaluated in
adherence to accreditation criteria. All Nurse Planners and key personnel who are in the process of
planning, implementing or evaluation an activity and have a question or a problem are to bring all issues to
me the PNP. In turn, I am accountable for ensuring that any issues relating to providing CNE are
addressed and adequately resolved. In the event that I am unable to resolve issues independently, I use
the resources within the organization to assist with the resolution process as needed.
An example of accountability for resolving issues is described in the following example:
Our provider unit was co-providing an educational activity with another organization who was a CME provider.
During the planning process for the educational activity, the Nurse Planner from our organization discovered
that the CME provider had independently selected all faculty for the educational activity, developed the
purpose and objectives, and had designed all the marketing materials. They wanted her to “sign off” on the
activity to award CNE credit for nurses. The Nurse Planner knew that this process did not adhere to
accreditation criteria but she felt intimidated by other members of the planning committee to sign off. She
came to me for advice. I validated that the planning process did not adhere to accreditation criteria and
worked with my Nurse Planner to clearly communicate the requirements of a CNE provider to the coprovider, the CME provider. I also helped her identify the steps to resolve the lack of adherence, ensuring
that the educational activity could still be provided as long as the steps were taken. The Nurse Planner then
spoke to the CME provider and outlined the steps that needed to be taken in order for the educational
activity to award contact hours. Feedback from my Nurse Planner was positive. The CME provider had not
understood the CNE process and was happy that the educational activity could still move forward as long as
the Nurse Planner was actively involved in the planning process. This did require some back-tracking
including sitting down and re-vetting the process for selecting faculty, developing the purpose and objectives
for the activity, and reviewing marketing materials as well as making changes to the educational activity that
ensured the learning needs of nurses would be met. Although it delayed the timeline for the educational
activity, both the Nurse Planner and CME provider felt the educational activity was stronger in the end.
SCORE: 3
 Rationale:
o The narrative answers the question
 PNP is accountable based on position/role in organization as well as personally taking
accountability for assisting NP and key personnel to resolve any issues
 Also cites taking accountability by using other resources to resolve issues if necessary
o Supporting evidence demonstrates that the criteria requirement is met
 Example includes details that demonstrate operationalization of the process description
SAMPLE RESPONSE 2:
The Primary Nurse Planner is accountable for resolving issues related to providing CNE due to the position of the
PNP in the organization. Since the PNP is the CEO of the organization, she is accountable for all CNE
activities.
For example, when anyone has a complaint about an educational activity, the CEO is responsible for resolving the
complaint.
 SCORE: 2
 Rationale:
o Narrative partially answers the question
 The narrative lacks detail to demonstrate how the PNP assumes accountability for
resolving issues related to providing CNE beyond accountability due to position/role as
both PNP and CEO
o Supporting evidence is lacking
 Fails to include a specific example to support the very weak narrative response
 Resolution of a complaint related to a CNE activity could demonstrate accountability of the
PNP if sufficient details were provided that linked accountability to the resolution process
3
Scoring Examples, Provider Units, Nurse Peer Reviewer Evaluation
8.29.12
Describe and, using an example, demonstrate each of the following:
EDP4. The process used to identify all actual and potential conflicts of interest for all members of the Planning
Committee, presenters, authors, and content reviewers.
SAMPLE RESPONSE 1:
Actual and potential conflicts of interest are evaluated closely by our Primary Nurse Planner. We have not had a
conflict of interest identified yet because we very thoroughly review every conflict of interest form for each
educational activity.
One example of how we evaluate conflict of interest is by closely examining the content chosen for educational
activities. If the content chosen is based on the best available evidence and there is no bias located in the
content or the slides for the educational activity, then we can be confident that we have evaluated the
educational activity before it is presented for potential conflict of interest. Then, we look at the evaluation
forms to determine if there is any bias that is reported. If anyone does indicate bias, we look at the
comments on the evaluation and follow up on each one that is reported.


SCORE: 2
Rationale:
o Narrative partially answers the question
 Lacks sufficient details to demonstrate process for evaluation beyond statement of “we
very thoroughly review every conflict of interest form”
o Evidence does not relate to the question or criteria requirement
 Fails to operationize process used to identify actual or potential conflicts of interest which
result from evaluation of relationships that an individual has with a commercial interest
organization
 Looking at the content of an educational activity and evaluating for bias is more
appropriately addressed under EDP9
 Concern about content integrity may be the result of an actual or potential conflict of
interest but is not an example of the process of identifying actual or potential conflicts of
interest
SAMPLE RESPONSE 2:
Although our organization has not identified an actual conflict of interest for a planning committee member or
presenter of an educational activity, we do have a process that we use within the organization to ensure
that we are thoroughly evaluating each person for an actual or potential conflict of interest.
Our current process for evaluating conflict of interest is as follows:
1. At the start of the planning process for an educational activity, the Nurse Planner documents whether she
has any actual or potential conflicts of interest related to the content of the educational activity. The
Primary Nurse Planner is responsible for reviewing her conflict of interest form and signing off that there is
no conflict of interest. The form that is used by our organization can be found under Tab B in the Appendix.
2. Once the PNP ensures that the NP has no actual or potential conflict of interest, the planning process can
continue.
3. The NP is responsible for reviewing all conflict of interest forms submitted by other planning committee
members, presenters, authors. We do not use content reviewers in this organization. The NP signs off on
each form to make sure that there is no actual or potential conflict of interest. Also, the NP monitors for
conflict of interest during the planning process.
4. The educational activity is presented. If possible, the Nurse Planner attends the educational activity and
monitors for any conflict of interest. Sometimes, other members of the planning committee monitor the
sessions.
5. The evaluation form for the educational activity includes a place to report any bias that is determined in the
activity and a comment section where the person can write in the bias that they identified. The Nurse
Planner and planning committee reviews each evaluation form after the educational activity to make sure
that there was no bias identified.
4
Scoring Examples, Provider Units, Nurse Peer Reviewer Evaluation
8.29.12
6. The planning committee discusses the educational activity and determines if there was any conflict of
interest found.
We do the same process for all of our educational activities and have not had any conflicts of interest reported.


SCORE: 2
Rationale:
o Narrative answers the question and responds to criteria requirement
o Supporting evidence is lacking
 No specific example provided to demonstrate process
 Regardless of whether an organization actually identified an actual or potential conflict of
interest, it should still be able to provide an example of the process used to identify actual
or potential conflicts
SAMPLE RESPONSE 3:
The process of evaluating for actual or potential conflicts of interest for members of the planning committee,
presenters, authors, and/or content reviewers is found in our organizational policy and included in the Appendix
under Tab F. The Nurse Planner for each educational activity is responsible for ensuring that any actual or
potential conflicts of interest are identified and resolved in the planning phase of the activity. In addition, when the
educational activity is presented, the Nurse Planner or her designee monitors for any actual or potential conflicts of
interest that may arise. All activity evaluation forms (see example included in the Appendix under Tab G and in
each educational activity submitted as Attachments A, B and C) include an item where learners can report any bias,
conflict of interest, or any other concern that should be brought to the attention of the planning committee.
An example of how the process of evaluating for an actual or potential conflict of interest is described below:
Our organization was planning an educational activity on the treatment of asthma in the pediatric patient. The
Nurse Planner completed a conflict of interest form and declared that she had no actual or potential conflict of
interest related to the content of the educational activity. The Primary Nurse Planner of our organization reviewed
the form to ensure that there was no COI and signed off on the form in the space for the reviewer. The Nurse
Planner then asked everyone on the planning committee to complete a COI form. Our COI form is found in the
Appendix under Tab E. The Nurse Planner reviewed each COI form and signed off in the space for the reviewer.
No COI was declared by any member of the planning committee. The planning committee then identified an expert
in the field of pediatric asthma from the local children’s hospital. The expert agreed to present for the educational
activity. When the expert completed the COI form, it was identified that the expert declared relationships with three
pharmaceutical companies. The Nurse Planner reviewed the COI form and determined that one of the
pharmaceutical companies made an asthma medication. The Nurse Planner and planning committee met to
discuss whether the expert had an actual or potential conflict of interest. The Nurse Planner and planning
committee determined that the relationship declared by the expert was relevant to the content of the educational
activity and that resolution would be required to ensure that the activity was presented without bias to the learners.
The options for resolution were reviewed by the Nurse Planner and planning committee members and they
included:
1. Removing the individual with conflicts of interest from participating in all parts of the educational activity.
2. Revising the role of the individual with conflicts of interest so that the relationship is no longer relevant to
the educational activity.
3. Not awarding continuing education contact hours for a portion or all of the educational activity.
4. Undertaking review of the educational activity by a content reviewer to evaluate for potential bias, balance
in presentation, evidence-based content or other indicator of integrity, and absence of bias, AND
monitoring the educational activity to evaluate for commercial bias in the presentation.
5
Scoring Examples, Provider Units, Nurse Peer Reviewer Evaluation
8.29.12
5. Undertaking review of the educational activity by a content reviewer to evaluate for potential bias, balance
in presentation, evidence-based content or other indicator of integrity, and absence of bias, AND reviewing
participant feedback to evaluate for commercial bias in the activity.
Because the expert from the pediatric hospital was considered a leader in the field of pediatric asthma, the Nurse
Planner and planning committee felt that it was important that the target audience had the benefit of hearing his
presentation. At the same time, they knew that they had to implement a process to resolve the COI. They chose to
implement resolution process #4 as outlined above.
First, they spoke to the expert and provided him with a copy of the speaker guidelines developed by the
organization (see Appendix under Tab H). They asked the speaker to submit his slides 2 weeks prior to the
presentation so that they could be reviewed by a content reviewer. Because the Nurse Planner and planning
committee members were not experts in the field of pediatric asthma and therefore could not effectively review the
content for potential bias, the Nurse Planner and planning committee identified a content reviewer to review the
slides. Before the content reviewer reviewed the presentation, she submitted a COI form which was reviewed by
the Nurse Planner. No COI was declared by the content reviewer or identified by the Nurse Planner. The content
reviewer reviewed the presentation and the content was found to be evidence-based and according to accepted
clinical guidelines. In addition, all drug names referenced in the presentation were generic. The content reviewer
signed off on the presentation as acceptable. During the presentation by the expert, the Nurse Planner remained in
the audience to ensure that the expert did not deviate from the presentation and introduce bias into the activity. No
bias was identified and the speaker was well-received. The Nurse Planner documented her findings for the activity
file.


SCORE: 3
Rationale:
o Narrative answers the question and responds to criteria requirement
o Supporting evidence demonstrates criteria requirement is met

Note: This response has potential to be a “4” demonstrating superior performance. There is insufficient
detail in this example to determine a score of 4 at this time, however. The response clearly demonstrates
adherence to criteria but may also demonstrate superior performance if these actions were beyond what
was baseline for this organization and/or demonstrated actions beyond what would be expected from a
“like organization.”
6
Scoring Examples, Provider Units, Nurse Peer Reviewer Evaluation
8.29.12
Value/Benefit to Nursing Professional Development. Evaluation data indicates how the Approved Provider Unit,
through the learning activities it has provided, has influenced the professional development of its nurse learners.
Describe and, using an example, demonstrate:
QO5. How, over the past 12 months, the Approved Provider Unit has enhanced nursing professional development.
(Refer to identified quality outcomes list in OO4.)
SAMPLE RESPONSE 1:
As an approved organization, we are committed to improving the professional practice of nursing. On a yearly
basis, our Approved Provider Unit collects yearly summative evaluation data from our activities. With the
inception of the new criteria, our Approved Provider Unit decided to determine what nursing professional
development outcome we want to evaluate for the next year. In 2011, we determined we wanted to
evaluate an improvement in nursing practice or nursing care delivery.
Our organization made a commitment to ensure that 20% of the educational activities that we provided in the past
12 months included an evaluation of an improvement in nursing practice or nursing care delivery. In 2011,
20% of our educational activities equaled 15 activities. Because evaluation of an improvement in nursing
practice or nursing care delivery cannot be assessed at the end of an educational activity and must be
assessed after the nurse learner has returned to the practice setting, this required that our organization
identify a way to contact the nurse learner after the activity was over. We chose to collect an email address
for each nurse learner participating in an educational activity that was designed to improve practice or care
delivery and we sent a survey out 3 months after the educational activity was presented. We did ask nurses
their permission to use their email address for the survey and let them know that we would contact them at 3
months. We also told them that their email address would be kept confidential and that the information that
we received from the 3 month survey would help us do a better job providing educational activities in the
future as well as provide evidence of the value of continuing nursing education. We used an electronic tool
to distribute the survey. Our overall response rate was 30% which was a little disappointing because we
really wanted to capture all of our nurse learner feedback but a 30% return rate is not bad compared to
expected survey response rates reported in the literature. When we analyzed the 3 month post-survey
responses from the educational activities designed to improve nursing practice or care delivery, we found
that most nurses implemented at least one change in the practice setting that resulted in an improvement in
practice or care delivery. The most frequently implemented change was using a clinical practice guideline
instead of their usual method of delivering care. The second most frequently implemented change was
ensuring that all patients received discharge planning instructions at a level the patient could understand.
One concern we identified was that 35% of nurses did not implement a change due to barriers within the
practice setting. As a result of this information, our educational activities in the future will include a
discussion of how to overcome barriers in the practice setting.


SCORE: 3
Rationale:
o The narrative answers the question
 Process description demonstrates link between strategic planning process used by the
Provider Unit and improving nursing professional development by using summative
evaluation data to evaluate an improvement in nursing practice or nursing care delivery
o Supporting evidence demonstrates the criteria requirement is met
 Example links how the Provider Unit used a strategy to link their educational activities to
collecting data that evaluate improving the professional practice of nursing
SAMPLE RESPONSE 3:
The Primary Nurse Planner and Nurse Planners in our organization held a strategic planning session to determine
how to promote nursing professional development. As a result, one strategic goal that we identified for 2011
was to promote leadership skills of registered nurses practicing in the field of neuroscience nursing. As a
continuing education Approved Provider Unit within a specialty neuro/rehab hospital, our strategic plan for
continuing education lines up with the mission and vision of the larger organization.
7
Scoring Examples, Provider Units, Nurse Peer Reviewer Evaluation
8.29.12
We chose to implement our strategic goal of promoting leadership skills by including leadership as one topic of
each educational activity provided in 2011. This was a challenge initially because most of our educational
activities were not designed to promote leadership. They were designed to improve the knowledge and
skills of neuroscience nurses. Over time, however, we found that incorporating leadership into each
educational activity was easier to do. We first communicated to our members that one of our strategic goals
was going to be promoting leadership skills. Then, for each educational activity that was planned, we had a
“leadership” section. Examples of topics that we presented under the leadership section included:
advocating for your patient in the neuroscience unit, communicating neuro changes in a patient to a new
resident physician and leading innovative care delivery. Our Nurse Planners really got excited about this
goal and got creative with ideas. We included a leadership item on our evaluation form to evaluate if
learners liked the leadership topics. We are going to aggregate the leadership score for each month and
see if it improved over the past year.


SCORE: 2
Rationale:
o Narrative partially answers the question
 Lack of detail demonstrating how the Provider Unit used data to evaluate its influence on
nursing professional development
o Supporting evidence is lacking
 Activities evaluated at the level of satisfaction therefore example fails to demonstrate that
nursing professional development was enhanced
 No description of how the Provider Unit evaluated data to determine how it has influenced
nursing professional development
8