ASQ/ASA Fall Technical
Conference
October 18, 2002
Ron King
Corporate Statistician
Patricia Bush
Corporate Director of
Outcomes Management
Pilot E Learning Seminar
 Pilot Project to test and evaluate the
feasibility of implementing web based
learning system in an international system
of hospitals
E Learning Seminar Objectives
Participants Will Be Able To:
 Understand the health care system
challenges that can be resolved using E
Learning
 Outline functionality and limitations
 Review a blood utilization case study that
reflects the process and content of an E
Learning statistical session
 Discuss “lessons learned” and pitfalls to
avoid when using E Learning
SYSTEM
CHALLENGES
RESOLVED WITH
THE USE OF
E LEARNING
SHC System Overview
 SHC is the largest philanthropic
organization in North America with no third
party payments or government funding
 Survival is based on realized gains from the
SHC Endowment fund
 “State of the Industry” health care is
rendered free of charge to patients and their
families
SHC System Overview
 22 hospital system (40-80 beds):
– 16 States (Honolulu)
– 3 Countries (USA, Mexico, Canada)
– Patients originate from the entire North
American Continent so over 15
languages/dialects spoken by staff
– 6 time zones
 Mission is three pronged: Patient Care,
Education/Teaching and Research
SHC System Overview
 Specialty Organization:
– Burn Care (4)
– Spinal Cord Injury (3)
– Orthopaedic Surgery (19)
 Organizational structure provides:
– System Wide Outcomes Team
– One Performance Improvement Director per
Hospital to support statistical analysis and data
management
SHC System Challenges to
Meetings
 “It takes so long to get stuff done, so many
people have input and I am emailed to
death with changes…”
 “Our conference calls are so hard to follow
when everyone is talking at once…”
 “Our system has 22 sites, we just can’t get
all those people together at one time…”
SHC System Challenges
 “I don’t have time…”
 “Travel costs are sky high, our budget is
stretched to the max... conferences,
training and education just have to go…”
E LEARNING
FUNCTIONALITY
AND LIMITATIONS
Logistical Issues Getting
Started
 Originally Planned for July 2002
 Protracted contract negotiations
 Agreement made to use product in last week
of August
 Invitations to attend sent with little notice
 Firewall issues
 Last minute vendor change
 Session Actually took place Friday, October
11th
A Partner With Functionality
 Cyber CE (Established in 2000):
– Internet-based Training & Education Company
– Based on successful Physicians Executive MBA
Program at the University of Tennessee
– Blended learning model that includes:
•
•
•
•
Self-paced content review
Testing & evaluations
Learner tracking and certification (as needed)
Live, interactive web conferencing across the world
Functionality
 CyberCE Learning Lab
• Customized & secured educational web sites
• Asynchronous, self-paced content delivery:
– Word documents
– PDFs
– Narrated PowerPoint slides
– Streaming video
Functionality
 CyberCE Online Conference Room
• Live, interactive, synchronous sessions
• Uses voice-over IP technology
• Content sharing tools include:
– PowerPoint slides
– Chat box
– Application sharing
– Whiteboard
– Web Safari
– Surveys
• Archiving recorded sessions for playback
Technology Support
 Staff services include:
– Establishment of E Learning goals
– Course development and deployment
– Evaluation design and implementation
– Student tracking and certification
– Presenter training
– “Help Desk” support
Program Limitations
 Presenter “observes the class” for activity,
can’t see participants in person
 Lag time between changes made to a shared
document and/or pointer movement and
visibility on participant screens
 Powerpoint presentations/shared applications
are not able to be used simultaneously
 Presenters and Participants must have
designated equipment and applications in
advance of the program
A CASE STUDY:
E LEARNING
STATISTICAL
TRAINING SESSION
Get Ready
Get Set
GO!
Readiness Assessment
 Perform a readiness assessment of your
system that includes:
– Learning culture assessment/computer literacy
– Computer access and configurations (Sound
Card, Windows 95+ and Internet Explorer+)
– Availability of microphone head sets; computer
with external speakers/microphone,laptop, etc.
– Availability of the presenter to participate in a
one hour training session
– Technical support to download Java script and
presentations (Java Applet with GIF files)
Program Set Up
 Collaborate with constituencies to identify
learning needs/session date and time
 Generate an invitation with: session
purpose, time involved, equipment needed,
participants, presenter, content,
objectives/anticipated outcomes and
instructions for download of Applet and
files (done the day before the program)
 Contact vendor to set up technical
requirements, email access passwords
Content Preparation
 Develop power point slides and an agenda
(if using one) for the presentation and
submit to the vendor so the information can
be “published”
 Develop shared documents that will be used
for the session and store them on the
presenters desktop (Excel, Word, etc.)
A CASE STUDY:
E LEARNING
STATISTICAL
TRAINING SESSION
Blood Utilization
Control Chart
Selection and
Interpretation
Agenda for Blood Utilization
 How should you look at this data?
 What Control Chart is appropriate for this
data?
 Why Use individual X Chart and mR
Charts?
– CT Ratio is a measurement
– Each month consists of one CT ratio
measurement
Agenda for Blood Utilization
 Building the Control Chart and mR Chart
– Creating with Formulas in Excel
– Creating using QI Macros
 Interpreting the Control Chart
– Data Meets Assumptions?
– Control Chart Shifts and Trends
– Points out of Control
Hospitals with Neither Laboratory nor Blood Bank on Site
Blood Utilization CT Ratios
mR Chart
(January 2000 - July 2002)
3.0
2.0
CT Ratios
UCL = 1.6
1.0
CL = 0.5
UCL - Upper Control Limit UWL - Upper Warning Limit
LCL - Lower Control Limit LWL - Lower Warning Limit
CL - Center Line
Month
Jun-02
May-02
Apr-02
Mar-02
Feb-02
Jan-02
Dec-01
Nov-01
Oct-01
Sep-01
Aug-01
Jul-01
Jun-01
May-01
Apr-01
Mar-01
Feb-01
Jan-01
Dec-00
Nov-00
Oct-00
Sep-00
Aug-00
Jul-00
Jun-00
May-00
Apr-00
Mar-00
Feb-00
Jan-00
0.0
Hospitals with Neither Laboratory nor Blood Bank on Site
Blood Utilization CT Ratios
iX Chart
(January 2000 - July 2002)
6.0
5.0
UCL = 4.4
UWL = 4.0
CT Ratios
4.0
CL = 3.2
3.0
LWL = 2.3
2.0
LCL = 1.9
UCL - Upper Control Limit UWL - Upper Warning Limit
LCL - Lower Control Limit LWL - Lower Warning Limit
CL - Center Line
Month
Jul-02
Jun-02
May-02
Apr-02
Mar-02
Feb-02
Jan-02
Dec-01
Nov-01
Oct-01
Sep-01
Aug-01
Jul-01
Jun-01
May-01
Apr-01
Mar-01
Feb-01
Jan-01
Dec-00
Nov-00
Oct-00
Sep-00
Aug-00
Jul-00
Jun-00
May-00
Apr-00
Mar-00
Feb-00
Jan-00
1.0
Month
7/1/02
6/1/02
1.6
5/1/02
2.4
4/1/02
3/1/02
2/1/02
1/1/02
12/1/01
11/1/01
10/1/01
9/1/01
8/1/01
7/1/01
6/1/01
5/1/01
4/1/01
3/1/01
2/1/01
1/1/01
12/1/00
11/1/00
10/1/00
9/1/00
8/1/00
7/1/00
6/1/00
5/1/00
4/1/00
3/1/00
2/1/00
1/1/00
CT Ratio
Hospitals with Both Laboratory and Blood Bank on Site
3.2
UCL=3.04
3.0
2.8
2.6
Data Points
UCL
A
CL=2.26
B
2.2
Average
B
A
2.0
LCL
1.8
LCL=1.48
1.4
Agenda for Blood Utilization
 When and How to Re-Calculate the Center
Line and Limits
– Change in Process Detected
– Deleting Old Data
– Excel Formula Built Graphs
– QI Macros Graphs
 Interactive Examples
– Recalculation
– Questions and Answers
Hospitals with Neither Laboratory nor Blood Bank on Site
Blood Utilization CT Ratios
iX Chart
(January 2000 - July 2002)
6.0
5.0
CT Ratios
4.0
UCL = 3.4
3.0
CL = 2.6
2.0
LCL = 1.8
UCL - Upper Control Limit UWL - Upper Warning Limit
LCL - Lower Control Limit LWL - Lower Warning Limit
CL - Center Line
Month
Jul-02
Jun-02
May-02
Apr-02
Mar-02
Feb-02
Jan-02
Dec-01
Nov-01
Oct-01
Sep-01
Aug-01
Jul-01
Jun-01
May-01
Apr-01
Mar-01
Feb-01
Jan-01
Dec-00
Nov-00
Oct-00
Sep-00
Aug-00
Jul-00
Jun-00
May-00
Apr-00
Mar-00
Feb-00
Jan-00
1.0
Month
4/1/02
3/1/02
2/1/02
1/1/02
12/1/01
11/1/01
10/1/01
9/1/01
8/1/01
7/1/01
6/1/01
5/1/01
4/1/01
3/1/01
2/1/01
1/1/01
12/1/00
11/1/00
10/1/00
9/1/00
8/1/00
7/1/00
6/1/00
5/1/00
4/1/00
3/1/00
2/1/00
1/1/00
LCL=1.891354048
7/1/02
6/1/02
5/1/02
CT Ratio
Hospital With Neither a Laboratory Nor a Blood Bank on Site
4.9
4.4
3.9
Data Points
UCL=4.422779562
3.4
2.9
CL=3.157066805
2.4
1.9
UCL
A
B
Average
B
A
LCL
Program Evaluation
1) Were the functions required to prepare for
the E-Learning program easy to complete?
2) Did it take you more than 1 hour to
prepare/train for use of the system?
3) Did it cost more than $10.00 per
participant to purchase the equipment
necessary for the call?
4) Would you recommend this training
methodology for statistical education?
Program Evaluation
5) During the training session, were there any
technical difficulties that negatively
impacted the session?
6) Did you feel that during the session your
ability to “interact” with the document
sharing component of the software was
effective and efficient?
7) When you “raised your hand” to speak,
were you acknowledged in a timely manner?
Program Evaluation
8) Did the “lag time” between the time you
spoke and the response from the instructor
cause the training to be less effective?
9) List at least two things that could be done
to improve this training system
10) What functions of the system added
and/or detracted from your ability to learn
using E-Learning?
Percent Responding (n=9)
PROGRAM EVALUATION
Yes
100
90
80
70
60
50
40
30
20
10
0
sy
a
E
No
No Answer
ep
r
P
hr
1
+
ep
Pr
d
iff
10
on
on
n
i
i
$
e
D
t
t
<
m
ac
ac
st
ch
r
r
m
e
o
t
e
o
s
t
T
C
ec
In
Di
R
y
g
el
La
uld
m
i
o
T
W
PROGRAM EVALUATION
 This was much easier than I anticipated
 The ability to answer questions and get
feedback was great
 What a great way to have interactive
training
 Loved the “out of office” function and for
clapping/smiling/raising my hand icons
 Need another button so the participants can
“ask the presenter” to modify pace
 Loved the “chat box”
LESSONS LEARNED
 Instructor needs to invest time in getting
comfortable with the program so that they
can adjust for lag time
 5 minute introduction for participants to all
buttons/icons with interactive practice is
useful
 Grouping participants by computer and
content literacy would help keep the
program moving at an appropriate pace
LESSONS LEARNED
 Test all connections 15 minutes prior to
program start
 Have a vendor facilitator present for the
first session to help the Presenter moderate
the activity of the group
 Interaction takes time, add 5-7 minutes to
each interactive agenda item
LESSONS LEARNED
 Unable to view powerpoint slides and work
in a shared application at the same time plan the presentation so that you can avoid
multiple jumps between program
applications.
 Automate the program evaluation using
either the voting system or survey system
for immediate feedback
 Avoid the tendency to go quickly in order
to allow screens to update on attendee PCs
Thank You!
Ron King:
rking@shrinenet.org
Patricia Bush:
pbush@shrinenet.org