Practice Improvement using Virtual Online Training (PIVOT):
A Gamified Approach to Medical Education
UCSF and UC Berkeley collaboration
Submitter’s name, title, and contact information
Maria Dall’Era, M.D.
Associate Professor of Medicine
Division of Rheumatology
University of California, San Francisco
533 Parnassus Ave. U 384
San Francisco, CA 94143-0633
Phone: 415-476-0783
Names of project leader(s) and team members
UCSF
Maria Dall’Era, M.D.
Michelle Lin, M.D.
Lidia Espino
Christy Boscardin, Ph.D.
Co-project leader, Associate Professor of Medicine, UCSF
Co-project leader, Associate Professor of Clinical Emergency
Medicine, UCSF
Professor of Medicine, Associate Dean for Admissions for UCSF
School of Medicine
Financial Analyst, UCSF
Assistant Professor of Medicine, UCSF Office of Medical Education
UC Berkeley
Kathleen Lu
Kai Hsieh
Joel Henderson
Tom Tsai
Bill Allison
Technical Project Manager and Principal Designer
Lead Developer
Technical Advisor
Platform and Engineering Manager
Architecture, Platforms, and Integration Director
The Lupus Initiative
Lisa Amaker
Director
David Wofsy, M.D.
Relevant URLs


Application website: http://The-PIVOT-Project.org
PIVOT Project highlight video: https://www.youtube.com/watch?v=ESAwa8Mnvbg
(shown at the national conference and launch of the Lupus Initiative’s curriculum “Taking
Action Against Lupus” on May 16, 2013 in Washington D.C., 4 minutes)
Page 1 of 5
PROJECT DESCRIPTION
Summary of the PIVOT project: http://The-PIVOT-Project.org
Computer assisted instructional models are growing in popularity in medical education. Many
were created, however, only to demonstrate the novelty of the new technologies available
without a thoughtful scholarly framework or road map. To address this, the Practice
Improvement using Virtual Online Training (PIVOT) Project thoughtfully recruited a
multidisciplinary, multi-institutional team of content, education, design, and technology experts
in 2012 to create an interactive educational platform tailored to the learner and based in
instructional theory. The PIVOT team was comprised of a diversely skilled group of members
from UC San Francisco, UC Berkeley, and the American College of Rheumatology’s Lupus
Initiative program.
In March 2013, the PIVOT team completed the development of two innovative, computer-based
simulation “games” allowing learners to assess and manage virtual patients with systemic lupus
erythematosus (SLE). This effort was federally funded by the U.S. Health and Human Services
Office of Minority Health and The Lupus Initiative. The Lupus Initiative is a national education
program with the overarching objective of reducing health disparities in SLE. A critical
component of this work is improving the understanding of how to diagnose SLE by recognizing
classic symptoms and signs, and learning how to distinguish among potential mimickers of SLE.
The PIVOT cases are publicly available at http://The-PIVOT-Project.org for free. The team of
experts in SLE, educational theory, research, programming, and user design experience
conducted iterative, design‐centered brainstorming sessions to come up with novel educational,
gaming, and interactive elements in the modules to optimize learner engagement and learning.
In July 2013, the free PIVOT cases were made available to the global public and promoted by The
Lupus Initiative. In September 2013, the PIVOT cases were deployed at UCSF in the preclinical
medical school curriculum in the Rheumatology block.
An Innovative Solution for Medical Education
"Our students have changed radically. Today’s students are no longer the people our
educational system was designed to teach." - Marc Prensky, On the Horizon
Today’s learners are “digital natives” who have grown up in the culture of Google, Facebook, and
Twitter. Traditional educational approaches no longer are effective. High schools and colleges
are increasingly “flipping the classroom”, which involves viewing videotaped online resources as
homework and doing traditionally-defined homework assignments as in-person classwork in a
facilitated-discussion environment. This has sparked the development of a multitude of
asynchronous, online educational materials, such that learners can learn on their own time
outside of the classroom. It is only a matter of time before medical education undergoes such
similar transformation.
Currently in the medical education world, high-fidelity simulation has garnered much attention
because of its ability to observe and assess learners directly and provide immediate,
personalized, debriefing feedback for improvement. This synchronous form of education,
Page 2 of 5
however, requires significant time and resources from the educators because often only 4-6
learners can participate at a time for each case. This also requires a team of simulation experts to
provide real-time, technical support during each case to ensure the adequate functioning of the
simulated patient mannequin.
An online, virtual-patient simulation platform provides a hybrid solution, fusing the best
elements from both asynchronous learning and high-fidelity simulation. In our PIVOT Project, we
have created and implemented an asynchronous, multimedia, choose-your-own-adventure
platform that allows learners to manage a virtual patient autonomously without harming an
actual patient. Through this branched storyboard approach, our platform tracks clinical
reasoning and diagnostic skills, as well as incorporates educationally sound principles, such as
active learning, debriefing, and feedback of individual performance. The interactive platform also
uniquely incorporates elements of gaming principles, such as patient exam videos to enhance
experiential immersion, and scoring metrics based on novel assessment tools such as our novel
“differential diagnosis slider” and script concordance test questions.
Figure 1: Sample screenshot of a PIVOT case, demonstrating the images and
videos available in examining the virtual patient
Page 3 of 5
Figure 2: The PIVOT project’s own novel “differential diagnosis slider” which measures the thought processes of the
learner regarding diagnostic pretest probabilities
Figure 3: Upon case completion, the learner is provided a detailed feedback and debriefing,
focusing on the key clues and critical findings that were evident in the case
Page 4 of 5
Technology utilized in the project





Language: Ruby
Framework: Ruby on Rails
Database: PostgreSQL
Additional Technologies: AngularJS, Bootstrap, jQuery, Sass, Compass, RSpec, Cucumber,
Capybara
Deployment: Heroku: cloud platform as a service (PaaS)
Timeframe of implementation




Dec 2012-Mar 2013: Rapid, iterative development of educational platform and 2 cases
Mar-Sep 2013: Alpha and beta testing of platform
Sep 2013: Implementation into the UCSF preclinical curriculum
Sep-Dec 2013: Analysis of learner feedback in planning phase 2 of PIVOT project
Objective customer satisfaction data
After we implemented the two PIVOT cases in the UCSF second year medical school curriculum,
we administered surveys to the students to assess student satisfaction with the cases and
request specific feedback on various aspects of the case design and content. The students were
uniformly impressed with and excited about the cases. They appreciated the novel, gamification
approach and the opportunity to learn about clinical reasoning in a complex autoimmune disease
such as SLE.
Examples of student comments are as follows:
“It was fun and incentivizing to hunt for key clues.”
“A really interesting and complex simulation. I really enjoyed the design and format of the case.”
“I like the videos/audios, and the opportunity to choose what aspects of the H/P I wanted to do.”
“I liked watching my points add up and then having the chance to see how things were scored;
picking my own lab values.”
“Fun! Lots of extra information that helped me to both evaluate my current knowledge and learn a
lot.”
“There was a lot of room for exploration.”
In addition to soliciting feedback from the students, we also received very positive feedback from
the medical director of the UCSF Medical School Rheumatology block, Dr. Mary Nakamura. UCSF
is in the initial stages of re-designing the medical student curriculum, and is interested in
utilizing additional PIVOT cases to contribute to this effort.
We are hoping to launch a second phase of the PIVOT program by using the student feedback to
enhance the first two cases and create new cases in other rheumatologic diseases. These cases
will represent the first time that online, interactive cases have been utilized in the second year
medical student curriculum.
Page 5 of 5