Scholarship in Education
Rose van Zuilen, PhD
Director of Faculty Development
Co-Director of Longitudinal Curriculum in
Geriatrics, Pain Management, and Palliative Care
Division of Gerontology and Geriatric Medicine
University of Miami Miller School of Medicine
Learning Objectives
Define educational scholarship and distinguish it
from other forms of scholarship
 Describe a broad range of activities that
constitute educational scholarship
 Determine existing outlets to publish
educational products
 Utilize effective ways to capture and document
the impact and quality of your scholarly work

Department of Medicine, Thomas Jefferson University
Typical Model of Scholarship

Scholarship of
Research
Clinical Care
Teaching
Service
Academic Advancement is often Slower
for Clinician Educators
Study at Johns Hopkins School of Medicine
 A single track for promotions
 The odds of being at a higher rank were 85%
less for academic clinicians and 69% less for
teacher clinicians than for basic researchers
 Rigor of promotion process affected by the
paucity of valid evaluation methods

Thomas PA et al.(2004) Acad Med 79(3): 258‐264
Expanding the Definition of Scholarship
 “What we need, then, in higher education
is a reward system that reflects the
diversity of our institutions and the
breadth of scholarship, as well. The
challenge is to strike a balance among
teaching, research, and service, a position
supported by two-thirds of today’s faculty
who conclude that, ‘‘at my institution, we
need better ways, besides publication, to
evaluate scholarly performance of faculty.’’
Boyer, 1990
Expanding the Definition of Scholarship
Discovery
Build new knowledge through traditional research
Integration
Make connections across disciplines and place specialties in a
larger context
Application
The bridge between theory and practical use. Aid society and
professions in addressing problems
Teaching
Effective communication of knowledge to learners
Glassick’s Scholarship Assessed
1) Clear goals – the educator explicitly states the
basic purposes for the work, and defines realistic,
achievable objectives, including desired goals and
outcomes.
 2) Adequate preparation – the educator shows an
understanding of relevant existing scholarship and has
skills and resources drawn from this research and
from prior experience to advance the project.
 3) Appropriate methods – in conjunction with the
material and the context, the educator chooses,
applies and, if necessary, modifies methods wisely.

Glassick’s Scholarship Assessed
4) Significant results – the educator achieves the
goals, and contributes notably to the field in a manner
that invites further exploration.
 5) Effective presentation – the educator uses a
suitable style and organization to present the work
with clarity and integrity in appropriate forums to
reach the intended audience.
 6) Reflective critique – the educator thoughtfully
assesses the work him/herself and uses the resulting
perceptions, along with reviews and critique from
others, to refine, enhance or expand the original
concept.

Association of American Medical
Colleges Consensus Conference
Group on Educational
Affairs (GEA) met in 2006
 Criteria for educational
scholarship
 Five categories

◦
◦
◦
◦
◦
Teaching
Learner assessment
Curriculum development
Mentoring and advising
Educational leadership and
administration
Issues Addressed by the GEA
1. What educator activities should be included
in each category?
 2. What types of evidence should be included
for review in the academic promotion process?
 3. How should a faculty member present the
activities and associated evidence in a
promotion/educator portfolio?

Issues Addressed by the GEA
Educators’ contributions can be judged through
the effective presentation of evidence
pertaining to the quantity, quality, and
engagement with the educational community
 Evidence of such engagements requires
educators to draw upon the literature and best
practices (scholarly approach) or contribute
to the medical education field (scholarship).

Category 1 - Teaching
Any activity that fosters learning, including
direct teaching and the creation of associated
learning materials
 Lectures, small groups, precepting, role
modeling, formative assessments and feedback,
online courses
 Creation of learner materials strictly for
enhancing one’s own teaching

Teaching
Documenting Quantity
 Document the frequency and duration of
teaching with a description of one’s role at all
levels (UME, GME, CME)
 Provide a list of instructional materials
authored with a brief description of their
purpose, format, and length
Teaching
Documenting Quality
 Learner’s evaluations of teaching performance
 Peer evaluations of teaching
 Teaching awards and honors
◦ Describe the selection process and criteria

Evidence of learning
◦ Pre-post assessments of learner performance
◦ Self-reported learning
Teaching
Documenting a Scholarly Approach
 Describe how one’s teaching is informed by the
literature (e.g., competence based education)
 Provide a comparative analysis of one’s own
teaching and teaching materials with “best
practices” in the field
 Reflect critically on one’s own teaching and on
feedback received by others noting the impact
on subsequent teaching activities
 Document participation in personal learning
Teaching
Documenting Scholarship
 Evaluations from conference presentations
 Inclusion of educational products in a
peer-reviewed repository
 Data demonstrating adoption by other
faculty
 References or citations to the product in other
peer-reviewed materials
Category 2 - Learner Assessment
All activities associated with measuring
learner’s knowledge, skills and attitudes
 Development: Identifying and creating
assessment process and tools
 Implementation: Collecting data
 Analysis: Comparing data with correct answer
key or performance standards
 Synthesis and interpretation: Interpreting and
reporting data to stakeholders

Learner Assessment
Documenting Quantity
 Provide information about the size and scope
of the assessment activity, one’s role, and the
number and level of learners assessed
Documenting Quality and a Scholarly
Approach
 Provide evidence that the assessment tool is
reliable, valid and informed by the existing
educational measurement field.
Learner Assessment
Documenting Scholarship
 Presentations on the assessment process or
outcomes to local audiences
 Peer reviewed or invited presentations and
workshops at professional meetings
 Acceptance of the assessment tool in a peerreviewed repository
 Assessment research presented at national
conferences or published
 Invitations to review assessment tools
Category 3 – Curriculum Development
Curriculum is defined as a longitudinal set of
designed educational activities that includes
evaluation
 More than one teaching session or presentation
 Can occur at any level of training
 Basic science course, clerkship, theme, rotation,
fellowship, CME course

Curriculum Development
Documenting Quantity and Quality
 For each curricular piece authored, provide a
description of the purpose, intended audience,
duration, design and evaluation.
 Note specific contributions (if co-authored)
 Provide learner reactions and ratings
 Show outcomes with graphic displays of
improvements over time when possible
◦ Course examinations, NBME subject scores, OSCEs,
performance observations
Curriculum Development
Documenting a Scholarly Approach
 Demonstrate that the curriculum design was
informed by the literature and “best practices”
◦ a curriculum designed to target nationally accepted
competencies
◦ The use of a well-accepted approach to curriculum
development (e.g., ADDIE)
Curriculum Development
Documenting Scholarship
 Peer review at the local level (curriculum
committee, accreditation reviewers)
 Peer reviewed or invited presentations
 Acceptance of curricular material to a peerreviewed repository
 List of institutions that adapted the curriculum
 Requests to provide consultation
 Invitations to conduct peer-reviews
Peer Reviews – Getting Involved

Poster sessions (local and national)
◦ Opportunities through colleagues and societies

Conference abstracts and papers
◦ Invited as a society member, referral, and volunteer

Journal articles
◦ Published author or recommended content expert
◦ Solicited or unsolicited applications

Educational products (e.g., MedEdPORTAL)
◦ Solicited applications … so, volunteer

Grants and awards
Things to Consider Before Signing Up as
a Reviewer
Make sure the journal is a good fit
 How many manuscripts does the journal expect
you to review – in what time period?
 Is there a specific format for reviews
 Do reviewers receive feedback?
 Is the review system open or closed

Sylvia LM & Herbel JL.(2001) Pharmacotherapy, 21(4)
Category 4 – Mentoring and Advising
Educators often serve as mentors and advisors
for their trainees and colleagues
 Mentoring implies a sustained committed
relationship from which both parties obtain
reciprocal benefit, whereas advising is usually
more time limited and one directional
 Documentation should describe the nature of
the relationship and one’s effectiveness in
helping mentees and advisees meet their goals

Mentoring and Advising
Documenting Quantity and Quality
 Provide a list of mentees with their position
and time invested (duration, hours)
 List mentees significant accomplishments
◦ Publications, presentations, grants, awards
Standardized evaluation of advising effectiveness
 Narrative comments or letters from mentees

◦ Solicited or unsolicited – Keep records
Mentoring and Advising
Documenting a Scholarly Approach
 Participation in professional development
 Use of mentoring strategies informed by
literature
 Design of a program guided by current evidence
 Leading of initiatives to improve institutional
mentoring and advising practices
Mentoring and Advising
Documenting Scholarship
 Secure competitive funding for program development
or innovative mentoring projects
 Conduct skills enhancement training
 Publish peer-reviewed mentoring guides
 Receive mentoring awards
 Convene a scholarly conference on mentoring
 Initiate a faculty learning community on mentoring
 Serve as a mentoring consultant
 Conduct mentoring research
Category 5 – Educational Leadership and
Administration
Exceptional educational administrators and
leaders achieve transform organizations in their
pursuit of excellence
 Their work's value is demonstrated through
ongoing evaluation, dissemination of results, and
maximization of resources.
 Leadership roles are varied

◦ Clerkship or course director, associate dean,
educational committee membership, educational
grant PI, conference organizer
Educational Leadership and Administration
Documenting Quantity and Quality
 Describe the quantity, nature and duration of
administrative and leadership roles
 Describe the change that occurred as a result
of your leadership (problems identified, goals
established, action taken)
 Provide outcomes (e.g., MCAT scores of
applicants have increased, clerkship evaluations
have improved, AAMC GQ ratings improved)
Educational Leadership and Administration
Documenting a Scholarly Approach
 Creatively design and evaluate improvements
◦ Show you are making revision based on local
feedback, best practices, external peer review
Document ongoing QI
 Use evaluation tools to measure outcomes

◦ Pre-post assessments, accreditation surveys, cohort
performance on national exams
Educational Leadership and Administration
Documenting a Scholarly Approach
 Demonstrate attainment of objectives or
benchmarks (AAMC GQ, course evals)
 Use a 360-degree evaluation with peer
comparisons, benchmarks, or external review
 Employ self-reflection informed by best
practices in the field
Educational Leadership and Administration
Documenting Scholarship
 List invited and peer-reviewed presentations
and visiting professorship presentations
 Document awards received (e.g., best paper)
 Provide a list of institutions that have adopted
an innovation
 Curriculum model published on MedEdPORTAL
 List resources obtained
◦ Foundation support, grants, internal awards
Funding Sources for Medical Education
Research Grants

Agency for Healthcare Research and Quality (AHRQ) Grants
◦ http://www.ahrq.gov/fund/grantix.htm

American Association of Neurology Educational Research Grants:
◦ http://www.aan.com/go/education/eduresearch

Amgen Foundation
◦ http://www.amgen.com/citizenship/IME_overview.html

Arnold P. Gold Foundation:
◦ http://humanism-in-medicine.org/index.php/programs_grants

Arthur Vining Davis Foundations:
◦ http://www.avdf.org/FoundationsPrograms/HealthCare.aspx

Association of Professors of Gynecology and Obstetrics (APGO)
◦ https://www.apgo.org/grants.html

Association for Surgical Education Foundation - Center for Excellence in Surgical
Education, Research and Training.
◦ http://www.surgicaleducation.com/mc/page.do?sitePageId=28551&orgId=ase
Funding Sources for Medical Education
Research Grants - Continued

AstraZeneca Medical Education Research Grants:
◦ http://www.astrazenecagrants.com/

Fund for the Improvement of Postsecondary Education (FIPSE):
◦ http://www2.ed.gov/about/offices/list/ope/fipse/index.html

The Henry J. Kaiser Family Foundation,
◦ http://www.kff.org/

HRSA-U. S. Department of Health and Human Services
◦ http://www.hrsa.gov/

Josiah Marcy, Jr. Foundation:
◦ http://www.macyfoundation.org/apply

National Institutes of Health
◦ http://grants1.nih.gov/grants/index.cfm

NBME Stemmler Medical Education Research Fund:
◦ http://www.nbme.org/research/stemmler.html
Funding Sources for Medical Education
Research Grants - Continued

NSF Directorate for Education and Human Resources:
◦ http://www.nsf.gov/dir/index.jsp?org=EHR

The PEW Charitable Trust:
◦ http://www.pewtrusts.com/

Pfizer Medical Education Grants:
◦ http://www.pfizer.com/responsibility/grants_contributions/medical_education_gra
nts.jsp

RSNA Foundation Radiology Education Grants:
◦ https://www.rsna.org/Education_Scholar_Grant.aspx
◦ https://www.rsna.org/Radiology_Education_Research_Development_Grant.aspx

The Robert Wood Johnson Foundation:
◦ http://www.rwjf.org/index.jsp

Society for Academic Continuing Medical Education, Research Grants in Continuing
Medical Education
◦ http://www.sacme.org/SACME_grants
For Promotion…
Educators can present evidence focused on one
category or in multiple categories
 Types of evidence may vary but documentation
should be quantitative and qualitative
 Breadth of engagement (local, regional, national,
or international) varies by faculty rank
 Display information in easily digestible format
using tables, figures, graphs

Staying Current – New Trends

MOOCS
◦ http://www.moocs.co/Higher_Education_MOOCs.html
Flipped classroom
 The Khan Academy https://www.khanacademy.org
 Using Social Media to Advance Scholarship and
Academic Careers

http://www.geripal.org/
http://www.eprognosis.org/
Making it Count
Write about what you do
 Two-fers, Three-fers
 Disseminate your work in multiple ways to
multiple audiences
 Create alignments between your activities
 Be an advocate for yourself
 Start a mock portfolio (Adobe Acrobat)
 Choose wisely

Promoting Scholarship at Your Institution
Develop methods to assess and document
scholarship
 Educate stakeholders (faculty, chairs, deans,
P&T committee members)
 Start a workgroup (faculty learning community)
to develop policies to broaden the definition of
educational scholarship
 Be a leader and a mentor

Key Points
All faculty and administrators, particularly those
who serve on promotion and tenure
committees, need to be aware of the principles
of educational scholarship
 Teaching can be a scholarly activity but
dissemination, peer review, and reflective
critique are needed for scholarship
 Document quantity, quality, and engagement
with the education community
 Promote scholarship by getting involved

A Bit of Good News
AMA MedEd Update May, 2013
Questions???