Classification of Protocol Complexity and Staffing Needs for MCRU
Final Report
Submitted To:
Ms. Cyndi Bower
Administrative Managing Director, MICHR Research Innovation/Clinical Research Support
Group
Clinical Manager, Michigan Clinical Research Unit
1500 E. Medical Center Drive
Ann Arbor, MI 48109
Ms. Amanda Silva
Central Lean Coach, UMHS Michigan Quality System
2101 Commonwealth Boulevard
Ann Arbor, MI 48105
Ms. Cindy Priddy
Central Lean Coach, UMHS Michigan Quality System
2101 Commonwealth Boulevard
Ann Arbor, MI 48105
Dr. Mark Van Oyen
Professor, Industrial and Operations Engineering
1205 Beal Ave.
Ann Arbor, MI 48109
Submitted By:
Sanjeev Muralidharan, Hyeon Kyun Nho, Christine Rockwell, Ashwin Varghese
IOE 481 Team 11, University of Michigan
Date Submitted: April 22, 2014
Table of Contents
Executive Summary ........................................................................................................................ 1
Introduction ..................................................................................................................................... 4
Background ..................................................................................................................................... 4
Key Issues ................................................................................................................................... 5
Goals and Objectives .................................................................................................................. 5
Project Scope .............................................................................................................................. 5
Methods........................................................................................................................................... 6
Data Collection and Analysis...................................................................................................... 6
Performed a Literature Search ................................................................................................ 6
Evaluated the Nursing Student’s Draft Tool ........................................................................... 6
Observed Tasks ....................................................................................................................... 6
Conducted Interviews ............................................................................................................. 6
Surveyed Clinical Staff ........................................................................................................... 6
Took Data at Huddle Meetings ............................................................................................... 7
Attended Huddle Meetings ..................................................................................................... 7
Attended Protocol Initiation Meetings .................................................................................... 7
Updating the Draft Tool .............................................................................................................. 7
Performing Pilot Runs................................................................................................................. 8
Findings........................................................................................................................................... 8
Learning about the Current State ................................................................................................ 8
MCRU needs a unique tool to classify protocol complexity .................................................. 8
Outside factors can influence a protocol visit ......................................................................... 9
Clinical staff feel understaffed and busiest in the morning .................................................... 9
Daily staff scheduling process has a low value-add time percentage ................................... 11
MCRU has a finite list of procedures that can be performed ............................................... 11
Protocol initiation process exists to familiarize MCRU with study team needs................... 11
Evaluating the Draft Tool ......................................................................................................... 12
Performing Pilot Runs............................................................................................................... 12
Conclusions ................................................................................................................................... 13
MCRU Experiences Higher Demand in Mornings ................................................................... 13
Updated Tool More Accurate than Draft Tool ......................................................................... 13
Recommendations ......................................................................................................................... 13
Use Scores Proactively ............................................................................................................. 13
Reference Time Component of Tool Output ............................................................................ 14
Encourage Uniform Visit Labeling across Protocol Initiation and Visit Scheduling Stages ... 14
Expected Impact............................................................................................................................ 14
References ..................................................................................................................................... 15
Appendix A: Draft Protocol Complexity Tool (Microsoft Excel version) ................................... 16
Appendix B: Clinical Staff Survey ............................................................................................... 17
Appendix C: Huddle Meeting Tally Sheet.................................................................................... 19
Appendix D: Updated Protocol Complexity Tool ........................................................................ 20
Appendix E: Current State Value Stream Map ............................................................................. 22
Appendix F: List of Procedures Performed by MCRU ................................................................ 23
Appendix G: Clinical Staff Survey Results .................................................................................. 24
ii
Appendix H: Clinical Nurse Specialist Interview ......................................................................... 28
Appendix I: Evaluating Draft Tool ............................................................................................... 29
Appendix J: Updated Tool Pilot Run ............................................................................................ 30
iii
List of Figures and Tables
Figure E-1. Understaffed survey question results ........................................................................... 2
Figure 1. Understaffed survey question results …………………………………………...………9
Figure 2. Busiest time of day survey question results .................................................................. 10
Table 1. Huddle Meeting Tally Sheet Results .............................................................................. 10
Table 2. Huddle Meeting Tally Sheet Results, removing days with zero tallies .......................... 11
Table 3. Pilot Run Results for Understaffed Days ........................................................................ 12
iv
Executive Summary
The Michigan Clinical Research Unit (MCRU) is currently finding difficulty in predicting the
number and specialties of clinical staff needed to support protocol visits on a daily basis due to
high variability in protocol complexity. MCRU believes that they have adequate clinical staff
overall; however, it is difficult to predict necessary daily staffing levels due to the variety of
protocol requirements. Therefore, MCRU needs a consistent mechanism to evaluate the
complexity of any protocol and the corresponding staffing requirements for protocol-specific
visits. This mechanism would allow MCRU to better predict staffing for a given day and plan to
provide the best possible service to study teams and participants.
The Administrative Managing Director of MICHR Research Innovation/Clinical Research
Support Group would like to know how to improve staffing predictability. In 2010, a nursing
student developed a draft protocol complexity tool that provides a complexity score and the types
of clinical staff needed for a protocol, but the tool was not validated. The Administrative
Managing Director asked an IOE 481 student team from the University of Michigan to evaluate
and update the draft protocol complexity tool to predict the number and type of clinical staff
needed on a day-to-day basis. MCRU would like to use the updated tool at their protocol
initiation meetings, which are used to set up a new protocol or restart an outdated protocol.
Methods and Findings
To determine how to evaluate and update the draft protocol complexity tool, the team completed
a data collection plan with ten methods.
Performed a Literature Search
The team conducted a literature search to analyze similar studies and draw inferences about their
relevance to this project. The team learned that MCRU is unique from other facilities in that each
protocol visit can have a different participant care and staffing requirements.
Evaluated the Nursing Student’s Draft Tool
The team evaluated the draft protocol complexity tool based on daily staffing schedules for
February 20 and 21, 2014. From the evaluation, the team found that the complex and simple
tasks were often assigned the same number of points. As a result, protocol visits different in
complexity called for the same number of registered nurses (RN) or medical assistants (MA).
Observed Tasks
The team observed the Nursing Supervisor create the daily schedule to understand how staff
scheduling works. The team learned that the initial process of schedule creation is short; however
a lot of time is required for reviewing and finalizing the schedule. The team also observed
clinical staff to determine how the workload varies. The team observed that staff was busiest in
the morning (8 a.m. – 12 p.m.).
Conducted Interviews
The team interviewed the Clinical Nurse Specialist at MCRU to understand her role in the
protocol process. The team interviewed three RNs to determine how much time RNs need to
1
perform every task on the comprehensive list of MCRU services.
Surveyed Clinical Staff
The team created a survey for the clinical staff at MCRU to quantify observations on complexity
of procedures and the clinical staff’s view of daily workload requirements. The results show that
all clinical staff surveyed felt understaffed (Figure E-1).
In general, do you feel understaffed? (select as many that
apply)
8
Number of staff
7
6
5
4
3
2
1
0
No, in general, I Yes, there are
Yes, the
Yes, there are Yes, there are
Yes, the
do not feel
too many
protocols are
too many
not enough
protocols are
understaffed.
protocols
unevenly
protocols that clinical staff too complex for
scheduled each distributed
require an RN. members that the amount of
day.
throughout the
work at MCRU. clinical staff.
day.
Figure E-1. Understaffed survey question results
(Source: MCRU Survey, Data Collection: February 27, 2014 – March 7, 2014, Sample Size: 7)
Took Data at Huddle Meetings
The team prepared a tally sheet for the clinical staff to discuss a number of issues each afternoon
huddle meeting. The results show that an average of 17.78% of huddle meeting attendees felt
understaffed, with a minimum of 0% and a maximum of 100%.
Attended Huddle Meetings
The MCRU staff holds two huddle meetings every day to look through the schedule and
determine if the workload is appropriate. The team attended huddle meetings to understand what
happens during the meetings.
Attended Protocol Initiation Meetings
The team attended three protocol initiation meetings because the updated protocol complexity
tool will eventually be used before and during these meetings. These meetings confirmed that the
updated tool will be helpful in assessing staff needs.
Updated the Draft Tool
The team updated the draft protocol complexity tool several times. The updated tool includes the
findings from interviewing the RNs and surveying the clinical staff. The team updated the draft
tool by utilizing Microsoft Excel. The user can check the necessary tasks performed, input
number of times a procedure is performed, and input time required for a procedure. The updated
2
tool provides a complexity score based on the information the user inputs into the tool. The
updated tool provides total staff time necessary and breaks this time down into RN and MA time.
Performed Pilot Runs
The team performed pilot runs and found the updated tool’s scoring is accurate. The tool details
the amount of time required for the protocol, split into RN and MA time. The tool also gives a
complexity score to the protocol and states if an RN is required for the visit.
Conclusions
The team drew two conclusions due to the findings from evaluating and updating the draft
protocol complexity tool.
MCRU Experiences Higher Demand in Mornings
The survey results show clinical staff feel busiest in the mornings (from 8 a.m. – 12 p.m.). The
team also observed staff was busier in the morning than in the afternoon. However, the huddle
meeting results indicate the average % of staff who felt understaffed was 17.78. This result may
be due to the fact that the huddle meeting data was collected in the afternoon when staff were not
as busy, and therefore did not feel understaffed when answering that question.
Updated Tool More Accurate than Draft Tool
Through the pilot runs, the team found that the updated tool is more accurate in predicting
staffing needs than the draft tool. The draft tool did not include many of the procedures that
MCRU performs. The draft tool also assigned the same score to procedures that were different in
complexity. The updated tool eliminates these two problems by providing a comprehensive list
of updated services and more accurately quantifying the different complexities of procedures.
Recommendations
Based on the findings from evaluating and updating the draft protocol complexity tool, the team
presents the following recommendations:
Use Scores Proactively
The team recommends MCRU use scores generated by the tool to proactively match workload to
staff available. This approach will improve staff utilization and reduce idle time and over time.
Reference Time Component of Tool Output
In addition, the team recommends that the time component of the output be used to help
determine the duration of the participant appointment. By allocating adequate time for visits,
MCRU can provide a higher quality of service to participants through reduced wait time before
the visit and idle time during the visit.
Encourage uniform visit labeling across protocol initiation stage and visit scheduling stage
The team recommends MCRU encourage study teams to use uniform visit labeling across the
protocol initiation and visit scheduling stages. Consistent visit labeling will be useful to tool’s
user, because visit numbers will be consistent in the tool’s output and daily staffing schedule.
3
Introduction
The Michigan Clinical Research Unit (MCRU) is the clinical research component of the
Michigan Institute for Clinical and Health Research (MICHR) and has the infrastructure to
support human clinical research. MCRU offers space, personnel, and specialized equipment to
study teams who perform human clinical trial studies involving outpatient and extended stay
procedures. The logistics needed to support each study is called a protocol.
MCRU is currently finding difficulty in predicting the number and specialties of staff needed to
support protocol visits on a daily basis due to high variability in protocol complexity. MCRU
believes that they have adequate clinical staff overall; however, it is difficult to predict necessary
daily staffing levels due to the variety of protocol requirements. Therefore, MCRU needs a
consistent mechanism to evaluate the complexity of any protocol and the corresponding staffing
requirements associated with protocol-specific visits. This mechanism would allow MCRU to
better predict staffing for a given day and plan to provide the best possible service to study teams
and participants.
The Administrative Managing Director of MICHR Research Innovation/Clinical Research
Support Group would like to know how to improve staffing predictability. In 2010, a nursing
student developed a draft protocol complexity tool that provides a complexity score and the types
of clinical staff needed for a protocol, but the tool was not validated. The Administrative
Managing Director asked an IOE 481 student team from the University of Michigan to evaluate
and update the draft protocol complexity tool. This report describes the project and is organized
as follows: background, key issues, methods, findings, conclusions, recommendations, and
expected impact.
Background
MCRU is currently supporting approximately 300 study teams who are conducting about 80
active protocols in a given month. Active protocols are those that are presently being supported
by MCRU and seeing at least one participant. MCRU can operate on a 24/7 basis when
necessary; however, the majority of protocols take place between 8 am and 8 pm. The protocols
vary from one another in their complexities. A protocol can be as simple as one blood draw, and
others can be as complex as a three-night study. The day-to-day staffing needs vary with the
types of protocols scheduled.
MCRU does not have a standardized method to schedule daily staffing or a quantifiable measure
to help predict the best daily staffing schedules. Currently, the MCRU Nursing Supervisor relies
on her experience and protocol knowledge to match staff with protocols. In addition, MCRU has
two daily huddle meetings (at 7:55 am and 3:15 pm every day) and relies on anecdotal evidence
reported by staff at these meetings to determine the effectiveness of staffing schedules.
MCRU management expresses the experience of ‘peak and trough’ demand for their services.
According to the MCRU management, on certain days, it appears that the staff is being
overworked while on other days they appear to be underutilized. Due to the high variability in
protocol complexity and the reported ‘peak and trough’ demand, MCRU finds it difficult to
4
efficiently match their available human resources, which includes five full time registered nurses
(RN), a part time RN, a full time medical assistant (MA), and a part time MA, to the protocols.
MCRU has not yet faced serious problems with the current scheduling method; however, they
want to quantify daily staffing needs in a standardized way to improve staffing predictability.
Potential consequences of not being able to predict staffing needed for protocols include
deviations in protocols (i.e. time sensitivity). Improving staffing predictability will also help
MCRU to plan for any future demand growth.
A nursing student previously developed a draft protocol complexity tool for MCRU to estimate
the protocol staffing needs by assigning a complexity score to each of the protocols. The
complexity score in the draft tool is calculated by assigning and summing scores for specific
procedures of each protocol. This complexity score indicates a protocol visit’s complexity and
types and numbers of clinical staff needed. The draft tool was not validated, and it was left
unused for four years.
MCRU asked the student team to evaluate and update the draft protocol complexity tool to
predict the number and type of staff needed on a day-to-day basis. MCRU would like to use the
tool at their protocol initiation meetings, which are used to set up a new protocol or restart an
outdated protocol.
Key Issues
MCRU needed this project due to the following key issues:
 Staff feel frustrated due to lack of predictability in staffing needs for protocol visits.
 Clinical staff experiences overutilization and underutilization due to different protocol
complexities and intensiveness of effort required.
 Integrity of a study team’s data could be compromised due to time sensitivity issues,
resulting from inappropriate allocation of clinical staff.
Goals and Objectives
The primary goal of this project was to evaluate and update the draft protocol complexity tool.
To achieve this goal, the student team addressed the following objectives:
 Standardize clinical staff-to-visit assignment
 Reduce errors in RN assignment
Project Scope
This project included the MCRU facility within the Cardiovascular Center (CVC) and the dayto-day staff scheduling of RNs and MAs. Protocols were analyzed by visits, and only outpatient
visits, which comprise 90% of MCRU’s business, were included.
The protocols requiring extended stays were not included in this project. However, MCRU plans
to incorporate all types of protocol visits into the tool later in the future. This project did not
involve protocols in offsite MCRU locations that are outside of the CVC building or the MCRU
2U mobile unit. It did not include monthly staff, participant visit, or equipment scheduling.
5
Methods
To evaluate and update the draft protocol complexity tool, the team completed ten methods to
collect and analyze the data. The following section discusses the data collection and analysis.
Data Collection and Analysis
To determine how to evaluate and update the draft protocol complexity tool and assign a
complexity score to protocol visits, the team developed a data collection plan. The plan included
performing a literature search, evaluating the draft tool, observing tasks, conducting interviews,
surveying clinical staff, taking data at huddle meetings, attending huddle meetings, attending
protocol initiation meetings, updating the draft tool, and performing pilot runs.
Performed a Literature Search
The team conducted a literature search to analyze similar studies and draw inferences about their
relevance to this project. The team looked at previous project final reports from IOE 481.
Evaluated the Nursing Student’s Draft Tool
The team evaluated the draft protocol complexity tool based on daily staffing schedules for
February 20 and 21, 2014. The team transferred the draft tool information from the paper version
of the draft tool into Microsoft Excel (Appendix A).
Every protocol has guidelines, created by MCRU staff, and a time and events table, created by
the study team. The team reviewed visits that were 8 a.m. – 8 p.m. each day and reviewed
protocol visit guidelines and time and events tables to see what procedures each protocol visit
required. The team used the guidelines and time and events table for a protocol to assign a score
to each visit using the draft tool.
Observed Tasks
The team observed the Nursing Supervisor create the daily schedule on January 28, 2014 to
understand the daily staff scheduling process. The team observed MCRU clinical staff for 30
hours to better understand daily operations. The team observed the staff on different days and
during different times of the day to determine how the workload of the staff varies.
Conducted Interviews
The team interviewed the Clinical Nurse Specialist at MCRU on February 12, 2014 to
understand her role and responsibilities during the protocol initiation process. The team
interviewed three RNs on February 25, 2014 and asked how much time RNs needed to perform
every task on the comprehensive list of services. The team interviewed RNs because RNs are
trained to perform all procedures that a protocol could require, whereas an MA is trained to
perform only a subset of the procedures.
Surveyed Clinical Staff
The team created a survey for the clinical staff at MCRU (Appendix B) to quantify observations
made by the team. Five RNs and two MAs completed the survey. Questions on the survey aimed
to assess how busy staff were during the day and if they felt understaffed. The questions also
6
aimed to assess complexity of the procedures performed by clinical staff and the impact of
charting in MiChart on workflow.
The team compiled the survey results in Microsoft Excel to analyze the responses and to
determine which responses were most common.
Took Data at Huddle Meetings
The MCRU staff holds two huddle meetings, at approximately 7:55 am and 3:15 pm every day to
look through the schedule and determine if the workload is appropriate. The team prepared a
tally sheet (Appendix C) for the clinical staff to discuss at each afternoon huddle meeting. The
leader of the afternoon huddle meeting facilitated discussion of the four categories on the tally
sheet and recorded data: number of protocol deviations, number of staff who felt overworked
during the day, number of specimens whose integrity was not maintained, and the number of
time sensitivity issues. The clinical staff collected this data for 10 to 30 protocols each day
February 10, 2014 – March 31, 2014.
Attended Huddle Meetings
The team attended five morning huddles and one afternoon huddle to observe problems that may
arise in scheduling.
Attended Protocol Initiation Meetings
The team attended three protocol initiation meetings on February 5 and 10, and March 17, 2014.
During these meetings, the protocol study team coordinates with all relevant MCRU
representatives to get approval on the logistics of the study. The team attended these meetings
because the updated protocol complexity tool will eventually be used before and during these
meetings to assess the complexity of protocol visits.
Updating the Draft Tool
The team updated the draft protocol complexity tool several times. The final version of the
updated tool is in Appendix D.
The team first updated the tool by adding procedures MCRU performs that were not on the draft
tool based on observations and interviews. The team also used input from interviewing the RNs
and surveying the clinical staff for procedure times and complexities, respectively.
The team updated the draft tool by creating a Microsoft Excel file. The file has three tabs:
Instructions, Tool, and Output. The user of the tool will select all the procedures involved in a
particular visit on the Tool tab.
The tool calculates required visit time and assigns a complexity score for a particular protocol
visit. Initially, required staff time for a particular procedure was calculated using the maximum
time reported by the RNs during the interview. The maximum time was used to provide an upper
bound on time. The team met with the Nursing Supervisor and based on her input changed any
procedure times that she stated were too long by making the corresponding updates to the tool.
The sum of the procedure times is the total time required for the visit. The tool gives the total
7
time required for the visit, and breaks this number into three times: time required by an RN, time
required by an MA and time during which the participant will be utilizing MCRU space.
The complexity of a procedure (1 to 5) was assigned based on the survey results of how complex
the staff perceived a procedure to be. To determine the complexity of each procedure, the team
analyzed the clinical staff survey results. Procedures reported as ‘Basic’ were given a complexity
of 1, ‘Moderate’ a complexity of 3, and ‘Complex’ a complexity of 5. Procedures reported as
both ‘Basic’ and ‘Moderate’ were given a complexity of 2 and similarly, procedures ranked both
‘Moderate’ and ‘Complex’ were given a complexity of 4.
The tool adds the complexity score of the procedures in a visit and put an ‘R’ or an ‘M’ at the
beginning of this score. An ‘R’ indicates that an RN is required to perform the at least one
procedure involved in the visit and an ‘M’ indicates that an MA can perform all the procedures
in the visit. This component of the complexity score is intended to reduce errors in RN
assignment.
The team added a button called ‘Save score information’ which saves the score and time outputs
to the Output tab. All the protocol visits the user uses the tool to score are stored in the Output
tab, allowing for a concise view of the time and complexity required for every visit.
Performing Pilot Runs
The team validated the updated protocol complexity tool by performing pilot runs. After each
pilot run, the team updated the tool based on the pilot run results and feedback from both the
Administrative Director and Nursing Supervisor.
The team performed two pilot runs on protocols on the February 20 and 21 schedules, the same
days that the team used to evaluate the draft protocol complexity tool. The team compared the
output of the both tools.
The team performed an additional two pilot runs using the protocols scheduled on two days
where all staff reported feeling understaffed on the huddle meeting tally sheet to see how well
the output compared with the staff schedule. The team performed these pilot runs using the staff
schedules for February 10 and March 6.
Findings
The team obtained results based on the evaluating and updating the draft tool from the methods
outlined above. These findings are stated in the following sections.
Learning about the Current State
The team had six findings about the current state of clinical staffing at MCRU.
MCRU needs a unique tool to classify protocol complexity
Based on the literature search, the team determined that many of the projects conducted by past
8
IOE 481 teams were related to optimizing the use of a scarce resource, such as a physician or a
patient room. In most of these cases, the handling procedure is similar for each patient. As a
research unit, MCRU is unique in that each protocol visit can have different participant care and
staffing requirements. This information indicated that the team should modify the draft protocol
complexity tool to specifically the procedures performed at MCRU. In addition, protocol visit
procedures had high variability, meaning MCRU may need to take additional iterations of
evaluating and updating the draft tool after the team completes this project.
Outside factors can influence a protocol visit
Based on a previous IOE 481 project, the team learned about outside factors that affect MCRU
staffing. MCRU depends on the Investigational Drug Service (IDS) to produce investigational
drugs for certain protocol visits [1]. If IDS does not produce the drug on time, this delay affects
the length of time the participant needs to remain at MCRU and can negatively impact clinical
staff and participant interaction due to long participant wait times as well as specimen integrity.
Extra factors, such as timeliness of drug production, need to be kept in mind when trying to
improve staffing predictability.
Clinical staff feel understaffed and busiest in the morning
Based on surveys, observations, and tally sheets, the team found the clinical staff feel
understaffed and busiest in the morning.
The results from the clinical staff survey show that all clinical staff feel understaffed (Figure 1).
In general, do you feel understaffed? (select as many that
apply)
8
Number of staff
7
6
5
4
3
2
1
0
No, in general, I Yes, there are
Yes, the
Yes, there are Yes, there are
Yes, the
do not feel
too many
protocols are
too many
not enough
protocols are
understaffed.
protocols
unevenly
protocols that clinical staff too complex for
scheduled each distributed
require an RN. members that the amount of
day.
throughout the
work at MCRU. clinical staff.
day.
Figure 1. Understaffed survey question results
(Source: MCRU Survey, Data Collection: February 27, 2014 – March 7, 2014, Sample Size: 7)
The survey findings show that the clinical staff feel most busy from 8 a.m. – 12 p.m. (Figure 2).
9
On average, what me of day do you feel most busy? (select
as many that apply)
8
Number of staff
7
6
5
4
3
2
1
0
8-10 am 9-11 am 10-12 11-1 pm 12-2 pm 1-3 pm 2-4 pm 3-5 pm 4-6 pm 5-7 pm 6-8 pm
pm
Time Period
Figure 2. Busiest time of day survey question results
(Source: MCRU Survey, Data Collection: February 27, 2014 – March 7, 2014, Sample Size: 7)
Based on staff observations, the team noticed that on average, the majority of the protocols were
often scheduled in the morning, and that staff was not as busy in the afternoons and evenings.
These observations were consistent with the survey findings.
The results from the tally sheets are shown below in Table 1. The huddle meeting results showed
17.78% of MCRU staff felt understaffed on average.
Table 1. Huddle Meeting Tally Sheet Results
(Source: Huddle Meeting Tally Sheet, Data Collection: Feb 10, 2014 – March 31, 2014, Sample Size: 36
days)
Average
Std. Dev.
Min.
Max.
Number of
deviations in
protocol today
0.67
0.99
0.00
4.00
Number of staff who
feel as though MCRU
was understaffed today
(%)
17.78
34.17
0.00
100.00
Number of
specimens whose
Number of time
integrity was not
sensitivity
maintained today
issues today
1.03
1.31
1.92
1.55
0.00
0.00
8.00
6.00
Table 2 shows the results, omitting the days when zero tallies were recorded. For each of the four
categories listed in Table 2, the number of days where at least one tally was recorded is 15, 10,
15, and 24 days, respectively.
10
Table 2. Huddle Meeting Tally Sheet Results, removing days with zero tallies
(Source: Huddle Meeting Tally Sheet, Data Collection: Feb 10, 2014 – March 31, 2014, Sample Size: 15,
10, 15, and 24 days, respectively)
Average
Std. Dev.
Min.
Max.
Number of
Number of staff who feel
deviations in
as though MCRU was
protocol today understaffed today (%)
1.60
64.00
0.91
35.42
1.00
20.00
4.00
100.00
Number of
specimens whose Number of time
integrity was not
sensitivity issues
maintained today today
2.47
1.96
2.33
1.52
1.00
1.00
8.00
6.00
The Administrative Director noted differences in prompts, even though the four categories
remained the same, depending on which staff member led the huddle meeting data collection on
a particular day.
MCRU plans to continue taking data at their huddle meetings, as a result of this project, for
quality control purposes. MCRU plans on asking similar questions about protocol deviations,
understaffed feelings, specimen integrity, and time sensitivity issues.
Daily staff scheduling process has a low value-add time percentage
The team observed the Nursing Supervisor create the daily schedule. This initial step takes
approximately 20 minutes. By observing this process, the team learned the steps required to
create the daily staffing schedule. Based on these observations, the team created a Value Stream
Map of the daily scheduling process at MCRU (Appendix E) to visualize the overall process. The
value add time percentage is 3.57% because, of the 7 hour lead time, only 15 minutes of the
process is value add time.
MCRU has a finite list of procedures that can be performed
From the draft protocol complexity tool and conversations with the clinical staff at MCRU, the
team formed a comprehensive list of services that an RN and/or MA can perform at MCRU
(Appendix F).
The survey shows how the nurses rank the complexity of each task and gives information on how
long the nurses spend entering participant information into the University of Michigan Health
System (UMHS) charting system, MiChart (Appendix G). Charting is a part of every visit and is
not considered separately in the updated tool.
Protocol initiation process exists to familiarize MCRU with study team needs
Through attending the protocol initiation meetings, the team learned how new protocols are
started and how revised protocols are restarted. UMHS and MCRU management and staff from
finance, clinical staff education, pharmacy, staffing, and the study team discuss questions about
what MCRU needs to do for the protocol. The team learned that complexities of the protocol
visits can require a lot of discussion, and predicting staffing for these visits is challenging. One
factor that makes predicting staffing challenging is specimen retrieval because most protocol
visits need specimens to be taken in the morning, sometimes even outside of normal MCRU
11
working hours (8 a.m. – 8 p.m.). Hence, the updated tool will prove useful.
From the interview with the Clinical Nurse Specialist (Appendix H), the team learned that she
gains background knowledge of the study and determines questions that need to be asked at the
protocol initiation meetings. After the protocol initiation meeting, the Clinical Nurse Specialist
provides guidelines to educate the clinical staff on procedures required by the protocol. This
interview helped the team understand the current process of what happens before and after
protocol initiation meetings.
Evaluating the Draft Tool
The team evaluated the draft tool by using it to assign complexity scores based on 30 protocol
visits. From the evaluation, the team found that the draft protocol complexity tool often assigned
the same number of points for complex and simple tasks. As a result, protocol visits different in
complexity called for the same number of staff. Additionally, the draft tool always assigned at
least one RN to a protocol visit. However, not all protocol visits require an RN and can be
handled by an MA.
As an example of how the draft tool evaluated protocols, the output for Protocol 2488’s baseline
visit is shown in Appendix I. The tool stated the visit is moderate, as it has been given a
complexity score of 5 points and calls for 1 RN and 1 MA to be assigned for it.
Performing Pilot Runs
Based on the pilot runs, the team found that the scoring using the updated protocol complexity
tool was robust and accurate. As an example of how the tool functions, the baseline visit of
protocol 2488 is shown in Appendix J. The tool details the amount of time required for the
protocol, split into RN, MA and space time. The tool also gives a complexity score to the
protocol and states that an RN is required for the visit.
The tool also provided an insight as to why staff might have felt understaffed on the days they
said so. The pilot run results indicate that the time allocated to a number of procedures was too
little for certain visits, thus causing the staff to feel overworked. Table 3 presents the results from
the two pilot runs performed on schedules when the clinical staff reported feeling understaffed.
Table 3. Pilot Run Results for Understaffed Days
(Source: Performing Pilot Runs, Data Collection: April 7 – 9, 2014, Sample Size: 36 days)
Schedule
Scheduled visit duration <= pilot
run output of required staff time
Total number of
visits completed
Percentage of times visit
duration <= pilot run
output required staff time
2/10/2014
3
14
21.43
3/6/2014
7
23
30.43
Additionally, when performing pilot runs, the team noticed that there is a lack of uniformity in
protocol visit labeling when comparing the visit number (or name) in the protocol’s time and
12
events table and the visit number (or name) in the staff schedule.
Conclusions
The team drew two conclusions due to the findings from evaluating and updating the draft
protocol complexity tool. MCRU may be understaffed in the mornings and the updated tool
performs better than the draft tool.
MCRU Experiences Higher Demand in Mornings
The survey results show MCRU staff feel busiest in the mornings (from 8 a.m. – 12 p.m.). The
team also observed staff was busier in the morning than in the afternoon.
Based on the survey results (Appendix G), all staff reported they are understaffed due to too
many protocols scheduled each day, too many protocols that require an RN, not enough clinical
staff members that work at MCRU, and too complex of protocols for the clinical staff. However,
the huddle meeting results indicated that on average 17.78% of staff felt understaffed. For 26 out
of the 36 days (72% of the time), zero people present at the huddle meeting reported feeling
understaffed. This result may be due to the fact that the huddle meeting data was collected in the
afternoon when staff were not as busy, and therefore did not feel understaffed when answering
that question.
Updated Tool More Accurate than Draft Tool
Through the pilot runs, the team found that the updated tool predicts staffing needs more
accurately than the draft tool. The draft tool only states how many staff are required for a visit,
while the updated tool provides the time needed for each type of staff, as shown in the findings.
Additionally, the draft tool did not include many of the procedures that MCRU performs. The
draft tool also assigned the same score to procedures that were different in complexity. The
updated tool eliminates these two problems by providing a comprehensive list of updated
services and more accurately quantifying the different complexities of procedures. In conclusion,
the updated tool significantly improves the results of the old tool.
Recommendations
Based on the findings from evaluating and updating the draft protocol complexity tool, the team
presents the following recommendations.
Use Scores Proactively
The team recommends the scores generated by the tool be used proactively to match workload to
the staff available. This approach is in contrast to the current approach where, at times, the staff
is asked last minute to work extra hours to cover all of the visits that day. When the working
hours are predictable, the clinical staff is much more likely to be satisfied on the job. This
approach will improve staff utilization and predictability, as idle time and over time are reduced.
13
Reference Time Component of Tool Output
In addition, the team recommends that the time component of the output be used to help
determine the duration of the participant appointment. The staff time component will in most
cases be shorter than the length of the visit because it is solely clinical staff time required by the
visit. The space time component of the tool can be referenced to better plan for visit duration. By
allocating adequate time for visits, MCRU can provide a higher quality of service to participants
through reduced wait time before the visit and idle time during the visit.
Encourage Uniform Visit Labeling across Protocol Initiation and Visit Scheduling Stages
The team recommends MCRU encourage study teams to use uniform and consistent visit
labeling across the protocol initiation stage and visit scheduling stage. Consistent visit labeling
will be most helpful to tool’s user, because the visit numbers will be consistent in both the tool’s
output and the daily staffing schedule. In general, uniform labeling will decrease confusion for
both the study team and MCRU staff.
Expected Impact
The team collected data from meetings, interviews, and observations to refine the time and
complexity requirements for each protocol task. The draft tool was evaluated and updated to
provide a score for each of these requirements. Through the updated tool, a combined rating can
be generated for a protocol visit. The tool will result in:




A quantifiable and consistent method to evaluate time and complexity for protocols
Better scheduling efficiency to prevent overutilization or underutilization of staff
Improved employee satisfaction
Increased quality of services to participants through better use of resources
For a given day with a pre-scheduled set of protocol visits, the overall task time and complexity
can be factored into building optimal staff assignments. Nurses in the unit will be able to
quantitatively understand the reasoning behind the day’s staffing assignments. Additionally,
when scheduling new protocols, the Nursing Supervisor can evaluate the visit’s score and time
needed to ensure that there is enough nursing capacity before adding the visit to that day’s
schedule.
In addition to the updated tool, MCRU will continue to track metrics at the daily huddle
meetings, based on the team’s method of taking data at huddle meetings. MCRU will record this
information for quality control within the unit.
14
References
[1] L. Baker, A. Chang, and A. Pollock. "Final Report on Project to Assess Comprehensive
Pharmacy Services for Michigan Clinical Research Unit (MCRU).” Practicum in Hospital
Systems: Ann Arbor, Michigan. Report. 16 Dec. 2010.
15
Appendix A: Draft Protocol Complexity Tool (Microsoft Excel version)
Basic Care
Score
Research
Your Score Score
1
1
1
1
1
1
1
2
2
1
1
1
1
Admission > 30 mins
Assessment > 30 mins
Set - Up > 20 mins
Teaching > 30 mins
Discharge > 20 mins
Emotional Care > 20 mins
EKG/ECG
Monitoring/Vital Signs
Q < 15 mins
Q 30 mins
Q hr
Q > 1 hr
Timed Walks
Questionnaires
1
1
2
3
1
2
1
1
Your Score
1 Single Blood Draw
PK/PD, Multiple Blood Draws
1
1-5 draws
2
5-10 draws
3
10-20 draws
4
> 20 draws
Specimen Collection
1
urine sample
1
nose/throat swab
1 Pregnancy Test
1 OGTT (oral glucose tolerance test)
* Shaded boxes = Tasks can be performed by MA
* If more than 3 shaded boxes selected, add MA to scheduled staff
< 7 pts
7-19 pts
20-25 pts
> 25 pts
Basic Care
Moderate
Moderate
Complex
Points
staff 1 RN, 1 MA
staff 2 RN, 1 MA
staff 3 RN, 1-2 MA
staff 4 RN, 1-2 MA
Interventions
Score
Phlebotomy
Score
Length of Study
0-3 hrs
4-10 hrs
11-15 hrs
> 15 hrs (overnight)
Normal Healthy Participants
Disease Specific Participants
Pediatric Participants
Isolation Precautions
General Information
Your Score Protocol #:
Type of Care (basic moderate, complex):
Your Score
1 IV Placement
2 Multiple IV Placements
2 Placement of PICC/Cath/Port
Invasive Procedure Assist
1
0-30 min
2
> 30 mins
1 Single Agent Infusion
2 Multiple Agent Infusions
1 Conscious Sedation
EKG/ECG
OGTT
PICC
PK/PD
Total
Points
Staff needed
Abbreviations
Electrocardiogram
Oral glucose tolerance test
Peripherally inserted central catheter
Pharmacokinetic/Pharmacodynamic
16
Appendix B: Clinical Staff Survey
1. What is your role? (select one)
2. In general, do you feel understaffed? (select as many that apply)
No, in general, I do not feel understaffed.
Yes, there are too many protocols scheduled each day.
Yes, the protocols are unevenly distributed throughout the day.
Yes, there are too many protocols that require an RN.
Yes, there are not enough clinical staff members that work at MCRU.
Yes, the protocols are too complex for the amount of clinical staff.
3. On average, what time of day do you feel most busy? (select as many that apply)
8-10 am
9-11 am
10-12 pm
11-1 pm
12-2 pm
1-3 pm
2-4 pm
3-5 pm
4-6 pm
5-7 pm
6-8 pm
4. What % of time are you busy during the average work day?
5. We are trying to better understand MCRU procedures. Rate the complexity (basic, moderate,
complex) of each of the following procedures. If you have never performed a task, select “I have
never performed this task.”
Basic
Moderate
Complex
I have never
performed this task
EKG/ECG
Physical Assessment
Timed Walks
Vital Signs
Blood Draws
Pregnancy Urine Test
Urine Sample
Endoscopy
Lumbar puncture
Infusion
Injection
Oral drug
17
(cont.)
Liver biopsy
Muscle/fat biopsy
Skin biopsy
Conscious sedation
Oral Glucose Tolerance Test
Pelvic exam
Placement of catheter
Port access
IV insertion
Mixed meal test
6. How many times on the average day does charting in Mi Chart disrupt your normal workflow,
if any? (Enter a number)
7. How much time (in minutes) on the average day do you chart in Mi Chart?
8. Enter any additional comments. (optional)
18
Appendix C: Huddle Meeting Tally Sheet
IOE 481: MCRU Huddle Meeting Tally Sheet
To be performed at 3:15 pm huddle meetings
Date
Number of deviations Number of staff who feel
in protocol today
as though MCRU was
understaffed today
Number of specimens
Number of time
whose integrity was not sensitivity issues
maintained today
today
3/3/14
3/4/14
3/5/14
3/6/14
3/7/14
3/10/14
3/11/14
3/12/14
3/13/14
3/14/14
3/17/14
3/18/14
3/19/14
3/20/14
3/21/14
3/24/14
3/25/14
3/26/14
3/27/14
3/28/14
2
19
Appendix D: Updated Protocol Complexity Tool
20
(cont.)
21
Appendix E: Current State Value Stream Map
22
Appendix F: List of Procedures Performed by MCRU
MCRU Service
RN 1
Basic Services
EKG/ECG
Physical Assessment
Timed Walks
Vital Signs
Estimated Time (minutes)
RN 2
RN 3
Average
15
60
10-15
2-3
15
45-60
15
5
15
60
15
10
15
57.5
15
5.8
Specimen Collection
Blood
Pregnancy Urine/Blood Test
Stool
Urine
5
2
-
15
2
-
5-15
5
5
5
10
3
5
5
Procedures
Endoscopy
Lumbar Puncture
60
45
60-120
45
60
30
70
40
depends
30
-
depends
45
30
-
depends
30-45
15-30
-
37.5
26.3
-
30
30
30-45
120
45
45
120
15
45-60
90
30
45
150
30
30
-
90
150
30
30
-
120
150
30
30
30
15
210
105
150
30
30
30
15
210
Drug Administration
Infusion (Including IV set up)
Injection
Oral
SC injection
Biopsy
Liver
Muscle/Fat
Skin
Other
Conscious Sedation
Oral Glucose Tolerance Test
Pelvic Exam
Placement of Catheter
Port Access
IV Insertion
Mixed Meal Test
23
Appendix G: Clinical Staff Survey Results
In general, do you feel understaffed? (select as many that
apply)
8
7
Number of staff
6
5
4
3
2
1
0
No, in general, I Yes, there are
Yes, the
Yes, there are Yes, there are
Yes, the
do not feel
too many
protocols are
too many
not enough
protocols are
understaffed.
protocols
unevenly
protocols that clinical staff too complex for
scheduled each distributed
require an RN. members that the amount of
day.
throughout the
work at MCRU. clinical staff.
day.
Figure G-1.
(Source: MCRU Survey, Data Collection: February 27, 2014 – March 7, 2014, Sample Size: 7)
On average, what me of day do you feel most busy? (select
as many that apply)
8
Number of staff
7
6
5
4
3
2
1
0
8-10 am 9-11 am 10-12 11-1 pm 12-2 pm 1-3 pm 2-4 pm 3-5 pm 4-6 pm 5-7 pm 6-8 pm
pm
Time Period
Figure G-2.
(Source: MCRU Survey, Data Collection: February 27, 2014 – March 7, 2014, Sample Size: 7)
24
What % of me are you busy during the average work day?
2.5
Number of staff
2
1.5
1
0.5
0
60
70
75-80
Response
80
95
Figure G-3.
(Source: MCRU Survey, Data Collection: February 27, 2014 – March 7, 2014, Sample Size: 7)
25
Table G-1.
(Source: MCRU Survey, Data Collection: February 27, 2014 – March 7, 2014, Sample Size: 7)
Procedure
EKG/ECG
Physical
Assessment
Timed Walks
Vital Signs
Blood Draws
Pregnancy
Urine Test
Urine Sample
Endoscopy
Lumbar
puncture
Infusion
Injection
Oral drug
Liver biopsy
Muscle/fat
biopsy
Skin biopsy
Conscious
sedation
Oral Glucose
Tolerance Test
Pelvic exam
Placement of
catheter
Port access
IV insertion
Mixed meal
test
Basic
Moderate Complex
I have never performed this task
0
7
0
0
0
4
7
1
0
1
0
6
5
0
0
0
2
2
0
0
7
5
0
0
2
2
0
0
2
0
0
3
1
0
0
0
0
4
0
5
4
0
1
5
0
1
5
2
2
2
2
2
0
0
5
4
0
1
2
2
0
0
4
3
1
5
6
1
0
0
0
1
0
0
0
5
4
7
0
1
0
2
2
0
0
6
1
0
26
How many mes on the average day does char ng in Mi
Chart disrupt your normal workflow, if any? (Enter a number)
Number of Staff
6
5
4
3
2
1
0
1-2
2
Response (in number of mes)
2-3
Figure G-4.
(Source: MCRU Survey, Data Collection: February 27, 2014 – March 7, 2014, Sample Size: 7)
How much me (in minutes) on the average day do you chart
in Mi Chart?
6
Number of staff
5
4
3
2
1
0
30-60
60
Response (in minutes)
90
Figure G-5.
(Source: MCRU Survey, Data Collection: February 27, 2014 – March 7, 2014, Sample Size: 7)
27
Appendix H: Clinical Nurse Specialist Interview
IOE 481 Clinical Nurse Specialist Interview
2/12/14, 10:30-11 am
Q1. What tasks are you specifically responsible for to prepare for a protocol initiation meeting?
Answer:
-First step: MCRU initiation email
-Review all information about study on eResearch
-Determine what MCRU is responsible for
-Gain background knowledge about study
-Determine what questions need to be asked
-Look at the time and events table: what MCRU is doing on unit (or if mobile)
-Checklist of discussion points
Q2. What tasks are you responsible for after a protocol initiation meeting?
Answer:
-Take everything from meeting
-Provide guidelines to staff and nurses
-2-week period to review
-Every clinical staff member needs to sign off on the protocol
-Expectations are high for staff
-2 other nurses can help Chris with writing up materials if workload is too high
Q3. Do you feel that it is burdensome trying to remember what goes on in all of the protocols?
How do you keep track of everything?
Answer:
-Impossible to keep track of everything; too much information
-Using reference tools and guidelines – very valuable
-Sometimes protocols seem more complex on paper than in real life
Q4. In your opinion, if a score were to be assigned to each visit in a protocol, is that something
that could be done during a protocol initiation meeting? Or do you think it should be done by
MCRU or by the study team prior to a protocol initiation meeting?
Answer:
-Might be too tedious to perform at protocol initiation meeting
28
Appendix I: Evaluating Draft Tool
29
Appendix J: Updated Tool Pilot Run
30