Scoring Guidelines for Section 1: Scholarship of Teaching
1. Philosophy
0-2 pts
A. Should be theory-based and actionable, (how the theory would be implemented).
B. Educationally sound theories and principles, modeling professional reasoning about the role of
teacher/student, and importance of learning environment
C. For a clinical teacher, creating a supportive learning environment/self-directed
D. Rubric
1. Reflective
2. Express educationally sound theories, principles and practices (do not have to “name”)
3. Evidence that the theories, principles and practices have been applied
2. Preparation
0-5 pts
A. Rubric - 5 aspects
1. Education Degree (Masters, etc.), intensive teaching certificate program or post graduate
program.
2. Formal Prep (Harvard Macys, teaching certificate, Scholars in education program).
3. Continuing professional development (reading articles/books, mentors, journal clubs).
4. Personal Reflection (reviewing student and peer evaluations).
5. Special Conferences/workshops/ professional meetings/Seminars of educational principles
(team learning).
3. Quantity and diversity (based on types of teaching and types of teaching)
0-18 pts
A. Rubric - 3 aspects
1. Gross Quantity
a. General estimate of time spend in education/research/administration (other than
educational administration). Need to look at total time spent in context to whole
professional practice
2. Breadth
a. Types of teaching activities:
i. Facilitation: problem based
xi. Seminars
learning or team based learning
xii. Journal clubs
ii. Web-based
xiii. Chairing a Dissertation committee
iii. Small Group
xiv. Clinical skills assessment (e.g.
iv. Lectures
OSCEs)
v. Clinical teaching
xv. Research skills assessment
vi. Laboratory teaching
xvi. Other direct types of student
vii. Distance learning facilitation
assessment (e.g. serving as an
viii. Mentoring students or peers
oral examiner)
ix. Assessment
xvii. Simulations
x. Student Advising
xviii. Other teaching activities
3. Diversity
a. # of different types of learner groups (levels/across departments/professions and # of
different learner groups) – not everyone will have the “opportunity” to teach all these
groups, so the diversity would be taken into context based on their opportunity to be
involved with each of these groups.
b. Types of learner groups:
ACADEMY OF MASTER TEACHERS | Application Scoring Rubric | edited 26 July 2016
1
i. Undergraduates/medical students
ii. Graduate students (masters and
doctoral)
iii. Interprofessional
iv. Post Docs
v. Fellows/Residents
vi.
vii.
viii.
ix.
x.
xi.
Faculty Development
Community education
Regional
National
International
Other learner groups
4.
Quality
0-20 pts
Evidence should be backed up in the supporting documents and be referenced in the portfolio summary
document
A. Outcomes: Evidence
1. Do they have teaching goals (the aim of the teaching activity)
2. Are goals specific, obtainable, and consistent with stated educational philosophy?
3. Degree to which teaching goals are accomplished
4. Context of teaching - based on quantity
5. Clear examples of evaluation scores
B.
Provide evidence of quality (quantitative and qualitative)
1. Are results meaningful?
2. Do they assess the degree to which the learners are successful?
3. Is the evidence strong and consistent over time?
4. Examples of Quality might include:
a. Student evaluations (residents, etc.)
b. Peer evaluations
c. Exam score performance
d. Student comments (brief, bulleted quotes)
e. Mentoring
i. Positive feedback
ii. Accomplishments of mentees
iii. Achievement records of mentees
iv. Mentees passing board exam
v. Letters from mentees
4. Exemplary Achievements (Global indicators of quality, i.e. # and variety of awards (i.e. Osler
Clinical Teaching Award, nomination for a teaching award)
C. Rubric
1. Strength of the evidence of teaching quality
2. Consistency of the evidence of teaching quality
a. Need to look at this section as a totality
i. Which quality indicators are evidenced and relate to teaching activity
ii. How many of the quality indicators (quantity)
iii. How many of the activities have quality indicators
iv. Do people have quality indicators of more than 1 type of teaching activity?
3. Should have specific indicators
4. Should be positive indicators
5. Look at degree of credibility
ACADEMY OF MASTER TEACHERS | Application Scoring Rubric | edited 26 July 2016
2
6. Multiple indicators + high #s
7. Frequency
8. Select from laundry list of evidence – this list is not meant to be inclusive:
a. peer evaluations
b. student evaluations
c. Course/program director evaluations (also in concise tabular summary)
d. Invitations to teach outside of one’s school
e. Repeat invitations to teach the same group or in the same course
f. Growth in quantity
g. Learners’ success (in terms of their publications, grants, etc.)
h. Quotes from supporting letters
i. Exam Score Performance
j. Graduate comments
k. Mentoring outcomes
5. Dissemination
0-5 pts
A. Dissemination is a goal of a master teacher, but not all teaching activities will have dissemination.
B. Basically, this section is to demonstrate that “Learned lessons in teaching are shared”
C. Should take into consideration the “context of the dissemination, i.e. is it national – at AAMC or CLEC,
regional, state, or local
D. Evidence of dissemination include:
1. Publications based on teaching methods
2. Invited presentations that are on teaching methods
3. Peer-reviewed and instructional materials (provide peer review and get peer review)
4. Examples of teaching materials being shared with others
Scoring Guidelines for Section 2: Enduring Educational materials
1. Philosophy/Goals 0-1 pt
A. Should be theory-based and actionable (how the theory on enduring educational material would be
implemented).
B. Educationally sound theories and principles of enduring materials
C. Rubric
1. Reflective
2. Express educationally sound theories, principles and practices (do not have to “name”)
3. Evidence that the theories, principles and practices have been applied to the materials
2. Preparation
0-2 pts
A. Rubric - 5 aspects
1. Education Degree (Masters, etc. related to educational materials).
2. Formal course taken for academic credit (e.g. instruction design theory, curriculum design).
3. Self directed continuing education for developing educational materials (professional meetings,
reading articles/books).
4. Personal Reflection of evaluations for improvement of materials.
5. Special Conferences/workshops/ Seminars on generating educational materials.
ACADEMY OF MASTER TEACHERS | Application Scoring Rubric | edited 26 July 2016
3
3. Quantity and Scope of enduring educational materials
0-6 pts
A. Rubric - 2 aspects
1. Type/Quantity of Materials
a. Textbook or textbook chapter
b. Manuals, technical manuals, Guidelines
c. Patient education materials
d. Vignettes used in teaching (print, video or electronic formats). These materials must be
disseminated to be considered enduring educational materials.
e. CE program materials
f. Review articles
g. Curriculum Guides
h. Published material specifically for educators
2.
4.
Other materials are considered enduring educational materials if they are: a) Documented use by
other faculty, and/or b) Used by a single faculty for different target audiences
a. Items used in learner evaluations (e.g., multiple-choice questions, OSCE or CPX cases,
vignettes for short answer questions, etc.)
b. Computer-assisted instruction distributed in CD or WWW formats
c. Audio-visual materials (e.g., instructional videos, audio tapes, slide sets)
d. Simulations created with computers and/or physical models
e. Tools to create enduring materials (e.g., template for creating cases)
f. Resources used in conducting workshops or other teaching sessions
g. Resources developed for community outreach (such as K-12, or collegiate education).
h. Software programs with technical manuals for research application
i. Website or other electronic educational tools
3. Types of Target Audience
a. Undergraduates
e. Patient education
b. Graduate students (masters and
f. Post Docs, Fellows and Residents
doctoral) and/or medical students
g. Faculty Development
c. Interprofessional
h. Peer/continuing education
d. Community education
i. Other learner groups
Quality 0-7 pts
A. Rubric 2 aspects. Evidence should be backed up in the supporting documents and be referenced in
the portfolio summary document
1. Goals: Evidence
a. Are goals specific and obtainable, goals consistent with stated philosophy?
b. Degree to which goals are accomplished
2. Evidence (quantitative and qualitative)
a. Does the evidence assess the degree to which the materials are successful?
b. Does the evidence come from a credible source
c. Are results meaningful?
d. Are there multiple indicators of high quality
e. Clear examples of evaluation scores. Examples might include:
ACADEMY OF MASTER TEACHERS | Application Scoring Rubric | edited 26 July 2016
4
i.
ii.
iii.
iv.
student evaluations (residents, etc.)
Peer evaluations
Comments (brief, bulleted quotes)
Exemplary Achievements (Global indicators of quality, i.e. # and variety of awards
(selected best book, article, CD, web site by a society)
v. Use of materials outside of one’s school
vi. Number of requests to use the materials
vii. Quotes from supporting letters
5.
Dissemination 0-4 pts
This section is to demonstrate that “enduring materials are shared” and should take into consideration the
breadth, depth and impact of the materials
A. Evidence of dissemination include:
1. Publications based on teaching methods
2. Invited presentations that are on the enduring materials
3. Peer-reviewed instructional materials (provide peer review and get peer review)
4. Citations for publications/hit statistics
5. Poster and/or presentations at professional meetings, include whether it was invited and where
the meeting was held
B. Examples of Educational Resources that may NOT be considered Enduring Educational Materials
1. A syllabus that is used principally by an instructor for his/her own course. This type of syllabus
might be described as part of a mini-portfolio for the Teaching and Evaluation category.
2. Sets of slides or power point presentations or recorded lectures prepared by one individual or
course for repeated use in one course.
3. Materials that require constant updating due to evolving standards or technological changes.
Examples: Materials from lectures on billing codes or emergent technology.
4. Materials that were not developed with the explicit objective to enhance learning, for example:
materials designed specifically to communicate results of research or patient care activities to
other scientists or to other health-care practitioners, respectively.
5. Self published materials: textbooks that do not have clear evidence of dissemination.
6. Course websites used solely for course management, communicative, or clerical purposes.
7. Published abstracts
8. Research seminar materials/power point presentations
9. Research Articles (not educational related).
Scoring Guidelines for Section 3: Educational Leadership
1.
A.
B.
C.
D.
Philosophy of leadership
0-4 pts
Should be theory-based and actionable, (how the theory is implemented).
Sound leadership theories and principles
Thought about role of leadership and the importance in educational process
Rubric
1. Reflective
2. Express sound leadership theories, (do not have to “name”)
3. Express sound leadership principles and practices
4. Evidence that the theories, principles and practices have been applied
ACADEMY OF MASTER TEACHERS | Application Scoring Rubric | edited 26 July 2016
5
2.
Preparation for leadership role
0-3 pts
A. Rubric
1. Leadership Degree (e.g. MBA, Educational administration, etc. related to leadership).
2. Formal Prep (ELAM (executive leadership in academic medicine) program certificate, business
administration certificate).
3. Special Conferences/workshops/ Seminars on leadership.
4. Continuing professional development (reading articles/books, mentors, journal clubs).
5. Personal Reflection (reviewing student and peer evaluations of leadership role).
3.
Quantity and diversity (based on amount of educational leadership and types of leadership roles)
0-5 pts.
Consider local, regional, national and/or international leadership roles in the following (but not limited to)
educational areas: Undergraduates, medical students, graduate students (masters and doctoral), faculty,
peers, interprofessional and community.
A. Rubric - 2 aspects
1. Quantity
a. General estimate of time spend in educational administration. Need to look at total time
spent in context to whole professional practice.
2. Diversity/Breadth
Types of leadership activity (but not limited to):
a. Program director (GME, Graduate school, longitudinal themes etc)
b. Department Chair
c. Section head
d. Nursing School Program Director – Undergraduate and Graduate
e. Track Administrator (Oversees the individual Tracks in each Nursing Program)
f. Course director
g. Course Coordinator – Faculty position in Nursing
h. Curriculum committee member
i. Curriculum committee chair
j. Selectives director
k. Electives committee director or co-director
l. Honors Research program director
m. Clerkship director
n. Organize workshops, symposia and conferences
o. Chair education committee for a society
p. Chair or member of study section for review for education based grants
q. Journal editor or associate journal editor (not editorial board member)
r. Leadership in Professional educational organization
s. Sustained record of peer review for an educational Journal
4.
Quality
0-6 pts
Evidence should be backed up in the supporting documents and be referenced in the portfolio summary
document
A. Outcomes: Evidence
ACADEMY OF MASTER TEACHERS | Application Scoring Rubric | edited 26 July 2016
6
B.
1. Do they have leadership goals (the aim of the leadership activity)
2. Are goals specific and obtainable, goals consistent with stated leadership philosophy?
3. Degree to which leadership goals are accomplished
4. Clear examples of evaluation scores (e.g. as a course director)
Provide evidence of quality (quantitative and qualitative)
1. Are results meaningful?
2. Do they assess the degree to which the leadership activities are successful?
3. See “g” for examples of evidence
4. Which quality indicators are evidenced and relate to leadership activity
5. How many of the quality indicators (quantity)
6. How many of the activities have quality indicators
7. Do people have quality indicators of more than 1 type of leadership activity?
a. Need to look at this section as a totality
b. Should have specific indicators
c. Should be positive indicators
d. Look at degree of credibility
e. Multiple indicators + high #s
f. Frequency
g. select from laundry list of evidence – this list is not meant to be inclusive:
i. peer evaluations
ii. student evaluations
iii. Course/program director evaluations (also in concise tabular summary)
iv. Exemplary Achievements (recognition of leadership e.g. awards)
v. Invitations to assume a leadership role
vi. Growth in quantity
vii. Comments from supporting letters of leadership role
viii. Letters of commendation
ix. Workshop evaluations
x. Recognition from group served
5.
Dissemination
0-2 pts
This section is to demonstrate that novel approaches to educational leadership are shared
A. Should take into consideration the “context of the dissemination, i.e. is it national –AAMC
(Association of American Medical Colleges) or CLEC (Clinical Laboratory Educators Conference),
regional, state, or local
B. Evidence of dissemination include:
1. Publications based on leadership methods
2. Invited presentations that are on leadership methods
3. Examples of leadership materials being shared with others
ACADEMY OF MASTER TEACHERS | Application Scoring Rubric | edited 26 July 2016
7