Standardizing Learning for Advanced Medical Students Through an Electronic
Learning Platform
PROPOSAL SUMMARY/ABSTRACT
We will create a comprehensive toolkit that will aid clerkship directors in designing
asynchronous electronic supplemental curricula for Foundations 2 and Career Launch
elective rotations that ensure students' exposure to the most common and important
subspecialty topics. The curricula will be case-based, interactive, review key principles of
foundational science within a clinical context, and incorporate learning elements
dedicated to systems-based practice and uncertainties/knowledge gaps within medicine.
BACKGROUND & SIGNIFICANCE
The fourth-year of medical school has been the focus of intense debate among medical
educational leaders, with some calling for its reform and others its removal (1-7). Fourthyear curricula vary by institution both in content and duration (3, 5) but perennially
emphasize elective rotations (3). Electives have the potential to meet many of the goals
medical students identify for the fourth-year, including preparation for future
training/practice, exposure to areas of interest, and career exploration (7), and Program
Directors from all specialties recognize the value of certain fourth-year elective
experiences in preparing students for intern year (2). To effectively serve these purposes,
subspecialty electives should be designed such that their educational goals are clear and
their curricula well organized and academically rigorous, therefore addressing the most
common concerns raised regarding the fourth-year of medical school (3). However, the
experience currently offered by subspecialty electives may be highly variable, with the
educational content defined solely by the types of cases that arise during rotating
students’ brief time with the team.
At UCSF, the new Bridges Curriculum offers students many opportunities for elective
clinical experiences during both Foundations 2 and Career Launch. However, these
rotations are highly variable in terms of their curricula and structure, with no expectation
that students will be provided with educational content beyond what they experience
clinically nor any assessments that provide a structure or guide for students’ self-directed
learning.
SPECIFIC AIMS
As one way to address the variability in clinical electives at UCSF, we will build a
comprehensive toolkit to help elective rotation leadership design electronic supplemental
curricula that facilitate the rotating students’ self-directed learning while ensuring that the
rotation’s educational goals are met. The overarching goal is to work toward the
implementation of uniformly robust curricula for clinical electives while still preserving
the unique clinical experience that each elective rotation offers; such curricula would
allow students to continue seeing interesting, unusual cases while also ensuring exposure
to the most common and important subspecialty topics.
Specifically, our toolkit will promote the development of interactive online, case-based
asynchronous curricula that would represent a new teaching and learning method at
UCSF, with the goal of providing educational content that integrates foundational
science into the clinical context, includes learning of systems-based practice, and
addresses the uncertainties and knowledge gaps in medicine. The toolkit will be
designed with a collaborative approach in mind, with the goal of being broadly
applicable across departments and disciplines within the School of Medicine and with the
potential for extension to other UCSF schools, such as the School of Pharmacy. By
meeting these objectives, the toolkit will facilitate the creation of elective curricula that
fit within the spiral learning that is the goal of the new Bridges Curriculum.
PROJECT DESIGN AND METHODS FOR IMPLEMENTATION
Project Design
The toolkit will include guides, templates, and timelines to aid in the design of electronic,
case-based, asynchronous educational content for medical students participating in
elective clinical rotations during the Foundations 2 and Career Launch portion of the
Bridges Curriculum. Kern’s approach to curriculum development will be utilized to
provide a framework for the toolkit.
Targeted Needs Assessment
In the first portion of the toolkit, we will create a guide to performing a multistep
elective-specific targeted needs assessment. As a first step, we will provide a list of
resources to consider when developing lists of potential module topics, as well as a
detailed description of various possible ways to assess student opinion of perceived gaps
in knowledge specific to the elective (through focus groups, surveys, etc). As a way for
toolkit users to narrow and prioritize the lists of potential topics they generate through the
above steps, we will next give instructions regarding how to undertake the Delphi
Process, a method for building consensus among experts; information provided will
include how to select expert participants, as well as timelines, templates, and draft emails
for the process.
Goals & Objectives
The next portion of the toolkit will include a guide for crafting various objectives for the
curriculum based on the content areas identified through the targeted needs assessment.
Sample objectives from the Infectious Diseases pilot curriculum will be provided as
examples. We will also suggest inclusion of objectives not immediately proceeding from
the needs assessment – including components of history-taking and physical exam
maneuvers particularly important for the subspecialty – as such objectives can be easily
incorporated into other content-specific modules.
Educational Strategies
Toolkit users will next be guided in the development of three to five case-based,
interactive modules that include content meeting the objectives defined earlier in the
process. As a first step, they will use simple templates to draft the case in text. The
toolkit will include guides for integrating the following components into each module:
-
Foundational Science: how to select and incorporate relevant and high-yield
content from Foundations 1 (including lectures, chapters, etc)
-
-
Uncertainties & Knowledge Gaps: how to provide resources that facilitate
students’ engagement in self-directed deeper exploration of subspecialty topics
while highlighting areas of uncertainty, controversy, and future inquiry at a level
appropriate for the targeted learner
Systems-Based Practice: how to emphasize salient points about high-value care
and quality improvement initiatives relevant to the subspecialty/module topic at
hand, and as well as how to include information about costs and potential adverse
events in module discussions of relative benefits and risks associated with tests
and therapies
Included along with these guides will ideas for creating interactive learning elements
within the modules. Electronic templates for building modules through the platform
Articulate Storyline will be provided, and links to the ID pilot curriculum modules will
be included as examples. As more specialties participate and build curricula, a broader
array of sample modules will be provided.
Implementation
The implementation portion of the toolkit will contain a number of resources to aid in
curriculum development. Potential timelines for the entire process will be provided,
constructed using information gathered during development of the ID pilot curriculum.
Email templates will offer potential ways for the rotation leadership to introduce and
explain the curriculum to students. The toolkit will also include a guide to developing
active learning sessions for consolidation of the curriculum’s key learning points, as well
as a description of how trainees – especially those involved in the HPE pathway – might
be involved in the design of the curricula.
Evaluation & Feedback
Within the toolkit, a guide to evaluation will be provided that suggests a multistep
process for assessing the subspecialty curriculum feasibility and impact. The first
component will involve tracking use of the system, including the number of times the
system is accessed by each user, the times of day, week, and month the system is utilized,
and the duration of time the system is used during each login period.
As a second component of the evaluation plan, a template/outline for a student survey
(based on AMEE guide 87) that investigates respondents’ knowledge of the subspecialtyrelated content included within the curriculum will be included in the toolkit. We will
recommend that elective rotations begin administration of the survey as both a pre- and
post-test immediately to all medical students participating in the specific elective rotation
– even prior to the development of their new electronic curriculum – in order to evaluate
the effect of its implementation. Within the survey, we will recommend questions related
to the learners’ perception of the appropriateness of the curriculum material and their
overall satisfaction with the electronic learning platform and elective. This information
will help rotation leadership direct further improvements to the curriculum and ensure it
remains learner-centered.
As a third component of the evaluation plan, we will provide a template/outline for a
faculty survey (based on AMEE guide 87) that assesses students’ application of new
knowledge in practice, as based upon the curriculum’s goals and objectives. In particular,
we will advocate for the development of questions that attempt to determine students’
knowledge regarding the key topics covered in the curriculum, their understanding of
high-value care as it applies to the subspecialty, and the degree to which they ask
questions focused on the knowledge gaps and controversies within the subspecialty.
Implementation
The toolkit project will begin with focus groups of various elective rotation directors
from different departments. The purpose of these focus groups – beyond generating
initial interest in the toolkit project – will be to gain a sense of:
--the educational needs of the electives not being met by the current educational
structure
--the utility of a cased-based electronic supplemental curriculum in the eyes of the
rotation directors
--the types of electronic modules the rotation directors can envision being useful
--the rotation directors’’ ideas regarding how they might build and incorporate such a
curriculum into their rotations
This information will be used to determine additional toolkit content that would be useful
to rotation directors, as well as to develop strategies for promoting the toolkit among
rotation directors once finalized.
After analyzing the information from the focus groups, we will develop a draft of all the
toolkit components and resources. The draft toolkit will then be utilized in the
development of a pilot electronic curriculum for the Infectious Diseases consult elective.
This pilot process will be used to improve the toolkit, provide key estimates and
timelines, and create guides for trouble shooting different portions of the process.
Ultimately, portions of the ID curriculum will be used as examples within the toolkit, to
be supplemented with examples from other rotations’ curricula as they become available.
One finalized, an email announcing the toolkit will be distributed to the educational leads
for each department within the School of Medicine, as well as the individuals identified
as the directors of elective rotations.
Although the initial scope of the toolkit project is focused on the School of Medicine, the
possibility exists for extension to other schools as well. The developed curricula might
contribute to SOM-GME, as residents rotating with various subspecialties might find the
information useful for refreshing their knowledge of unfamiliar topics or reviewing
related aspects of foundational science in a clinically relevant context. The School of
Pharmacy might also be able to utilize the toolkits in developing modules for their
clinical learners, so we will plan to involve their leadership early in our project in order to
ensure the toolkits are more broadly useful.
EVALUATION PLAN
Initial evaluation of the toolkit project will be based on the number of specialties that
utilize the resource, as well as the number of curricula that are developed as a result. We
will also develop a questionnaire for the key contact within each subspecialty that helps
identify the strengths and weaknesses of the toolkit, the time and person power required
for curricula development and implementation, the intended and realized use of the
resultant curricula, and any barriers encountered. We will use this data to improve the
toolkit over time, as well as to evaluate its utility and success.
PLAN FOR SUSTAINABILITY
DESCRIPTION OF THE DIFFERENCES BETWEEN THE CONCEPT AND THE
FINAL PROPOSAL
TEAM MEMBERS
1. Brian Schwartz, MD (Associate Professor, Department of Medicine, SOM) (PI)
2. Emily Abdoler, MD (Resident Physician, Department of Medicine, SOM)
3. Christian Burke, BA (Director of Technology Innovations, Technology Enhanced
Education, SOM)
4. Patricia Cornett, MD (Professor and Associate Chair of Education, Department of
Medicine, SOM; Faculty Lead, Bridges Foundations 2 Design Committee)
5. Jenny Crawford, MA, MPH (Program Specialist, Bridges Curriculum, SOM)
6. Christopher Moriates, MD (Assistant Professor, Department of Medicine, SOM;
Lead for the Center for Healthcare Value’s “Caring Wisely” Program)
7. Health Professions Pathway Learners (MS4), TBD
8. Karen Hauer, MD, PhD (Professor of Medicine and Associate Dean for
Competency, Assessment and Professional Standards, SOM)
TOTAL BUDGET
$15,960
Salary Support:
- Brian Schwartz, MD (PI): As the Gold-Headed Can Endowed Teachign Chair for
the Department of Medicine, Dr. Schwartz has chosen to use funds from his
Endowed Chair to support his mentorship time (5%) for this project and would
prefer to apply for salary support to support a fellow interested in medical
education scholarly work.
- Infectious Diseases Fellow in the Clinician Educator Pathway: We are requesting
$15,960 for 20% support of slaray and benefits for a second-year infectious
diseases fellow. Approximate salary (including $10,000 annual housing
allowance) will be ~$72,000 plus ~$7,800 in health insurance benefits. The time
provided by the 20% salary support would enable the fellow to design and
conduct surveys of faculty and learners, learn to use Articulate Storyline II, learn
to use Canvas, create 3-5 online modules for the ID pilot project requiring
consultation with relevant experts, create and implement project evaluation,
design the toolkit and all included resources, engage in the Health Professions
Education Pathway, write up the project as scholarly work for presentation at
education meetings and submit for publication, At present, UCSF Infectious
Diseases Fellows receive the majority of their salary from NIH funded training
grants (T-32s) during their second and third years of training. Unfortunately, these
training grants are not structure to support medical education scholarly work and
therefore funds are not available to support fellows interested in such scholarship.
Therefore, we are hopeful that the Innovations in Education Funding Grants
Program will be able to support the development of a future subspecialty medical
education leader.
CONSULTATIONS COMPLETED
REFERENCES
1. Slavin SJ, et al. Curriculuar reform of the 4th year of medical school: the colleges
model. Teach Learn Med 2003;15:183-93.
2. Lyss-Lerman P, et al. What training is needed in the fourth year of medical
school? Views of residency program directors. Acad Med 2009;84(7):823-29.
3. Walling A, Merando A. The fourth year of medical education: a literature review.
Acad Med 2010;85:1698-1704.
4. Cianciolol A, Lower T. Implementing a student-centered year 4 curriculum; initial
findings. Med Ed 2012;46:1107-08.
5. Cosgrove EM, Ryan MJ, Wenrich MD. Empowering fourth-year medical
students: the value of the senior year. Acad Med 2014;89(4):533-35.
6. Sklar DP. Making the fourth year more meaningful. Acad Med 2014;89:527-28.
7. Wolf SJ, et al. Students’ perspectives on the fourth year of medical school: a
mixed-methods analysis. Acad Med 2014;89:602-07.
8. Cook DA, et al. Web-based learning in residents’ continuity clinics: a
randomized, controlled trial. Acad Med 2005;80:90-97.
9. Duque G, et al. From the facts to the screen: a blended model of teaching basic
hospital skills to 2nd year medical students. Medical Teacher 2006;28:729-33.
10. Cook DA. Web-based learning: pros, cons and controversies. Clin Med
2007;7:37-42.
11. Cook DA, et al. Internet-based learning in the health professions: a meta analysis.
JAMA 2008;300:1181-96.
12. Khogali SEO, et al. Integration of e-learning resources into a medical school
curriculum. Med Teach 2011;33:311-18.
13. Duque G, et al. Evaluation of a blended learning model in geriatric medicine; a
successful learning experience for medical students. Australasian Journal of
Ageing 2013;32(2):103-09.
14. Kumar AB, et al. Implementing a hybrid web-based curriculum for an elective
medical student clerkship in a busy surgical intensive care unit (ICU): effect on
test and satisfaction scores. J Surg Ed 2013;70(1):109-16.