Faculty Development
Issues in Evaluation






Definition
Types of Programs
Types of Evaluation Done
Evidence of Success
Believed Utility
Reason for Continuation of programs
• Need / Evidence of Success


Risks to current evaluation methods
Impetus for future work
Faculty Development Definition

activities designed to
improve an individual’s
knowledge and skills in
areas considered essential
to the performance of a
faculty member……
McLeod, Steinert, Nasmith, Conochie, CMAJ-JMAC, 1997
Types of Faculty Development




Instructional development
Professional development
Leadership development
Organizational development
Irby, 1995, from Berquist and Phillips
Faculty Development
Schemes for Program Evaluation:
 The Faculty
Teacher Attitudes/Knowledge/Behaviors
Pedagogy
Pedagogical Content Knowledge
Content Areas, e.g., End of Life, Geriatrics,
Organizational Leadership, Professional Skills
 Their Students
 The System
Faculty Development
Schemes for Program Evaluation:




Reaction (customer satisfaction)
Faculty Learning (change in Knowledge,
Skills, Attitudes)
Behavior Change (result of program)
Results (change in the system)
Blumberg, Deveau, Medical Teacher, 1995
Faculty Development
Schemes for Program Evaluation:




Academic dissemination
(presentations/publications)
Product development (materials/workshops)
Implementation ( #’s trained, maintained
and disseminated)
Institutionalization
Blumberg, Deveau, Medical Teacher, 1995
Faculty Development
Schemes for Program Evaluation:




Process Evaluation (participant ratings)
Content Evaluation (Change in knowledge,
skills, attitudes)
Outcome Evaluation (long term change)
Impact Evaluation (Impact on the system)
Holly, 1992
Faculty Development
Schemes for Program Evaluation:
 The Faculty
 Their Students
 The System
Faculty Development in Canada
Evaluation methods
3 outcomes:
 performance
 cognitive learning
 self-assessment re teaching skills

Outcome measures – positive
direction, but without statistical
significance, but small N.
Nasmith, Steinert, Saroyan, Daile, Franco, TchLng Med, 1997
Lessons from Faculty
Development in Clinical Teaching


Research Evidence for Benefits
Teachers and learners
– POSITIVE RATINGS OF EXPERIENCE
– IMPROVED EDUCATIONAL OUTCOMES FOR
FACULTY - knowledge, skills, and attitudes
•
•
•
•
SELF-RATINGS
RATINGS OF BEHAVIORS ON VIDEOTAPES
KNOWLEDGE ACQUISITION BY TEACHERS
IMPROVED STUDENT RATINGS OF TEACHERS
– IMPROVED OUTCOMES FOR LEARNERS -MOSTLY IN HIGHER ED LITERATURE*

Institutional / Societal Benefits ?
*Murray in Perry and Smart, 1997
Effective Teaching in Higher Education
Stanford Faculty Development Program
Faculty Development Outcomes


Reviewed different types of programs in
family medicine
• Fellowships
• Workshops
Results
• Changes measured, but no significant
performance changes
• Few publications
• Different needs for different faculty, tenure
vs non-tenure
Reid, Stritter, Arndt, Fam Med, 1997
Faculty Development
Schemes for Program Evaluation:
 The Faculty
 Their Students
 The System
AAMC - 2000
Faculty Development
/Faculty Affairs Offices:
76 of 125 institutions ->
93 faculty affairs and 21
development offices
Moahan, et al, Acad Med, May 2002
Faculty Development - Evaluation
Faculty development
offices focused on
use of services and
mentoring
Moahan, et al, Acad Med, May 2002
AAMC - 2000
Faculty Development
/Faculty Affairs Offices:
76 of 125 institutions ->
93 faculty affairs and 21
development offices
Moahan, et al, Acad Med, May 2002
Faculty Development in Canada



1986 – positive commitment – poor faculty participation
Recommendations:
• establish committees
• fund activities
• Recognize participation in promotion/tenure
1986-96: Major changes:
• Health care financing
• Downsizing
• Inpatient -> outpatient
• Multi-institutional and multidisiplinary care
• Curriculum reform
– Less stress on information acquisition
– Problem solving / interpersonal skills/attitudes
McLeod, Steinert, Nasmith, Conochie, CMAJ-JMAC, 1997
Faculty Development in Canada


All 16 schools responded
16 with Faculty Development Program vs 12 in 1986
• Sabbatical programs
• Grants to improve instruction/ courses, with major
emphasis on information technology/electronic media
• Awards for teaching excellence







Increasing feedback on teaching from faculty or
administrators (13/16 schools)
Systematic review of teaching by administrator- 6
schools
Senior faculty work with junior faculty – 2 schools
Workshops on management / research / scholarship
Fac Devel committees in 9 schools
½ of schools indicate inadequate budget
Outcomes reported at 4 schools
Stanford Faculty Development Program
McLeod, Steinert, Nasmith, Conochie, CMAJ-JMAC, 1997
Faculty Development in Canada

Most effective practices
•
•
•
•
•

Teaching instruction
Use of computers/information mngt
Sabbatical leaves
Teaching Awards
Inexpensive faculty development
Local faculty development
stimulated by clinicians with M Ed
Degrees
Stanford Faculty Development Program
McLeod, Steinert, Nasmith, Conochie, CMAJ-JMAC, 1997
Faculty Development in Canada

Stimuli for change
• Curricular change, needing more
tutors
• Curricular change – Uncertainty in
faculty
• Changes in health care –
examination of career and personal
goals
McLeod, Steinert, Nasmith, Conochie, CMAJ-JMAC, 1997
Faculty Development in Canada
 Prognosis
- - Good;
Collegiality and Institutional
Support
 Problem – lack of evaluation
on impact
 Need to identify outcomes and
methods
Stanford Faculty Development Program
McLeod, Steinert, Nasmith, Conochie, CMAJ-JMAC, 1997
Lessons from Faculty
Development in Clinical Teaching
Research Commentary
– Hard to do controlled experiments
– Few measures of pre-post change
– Few measures of long-term change
– Mostly participant ratings
– Few measures of actual behavioral
change in faculty
– Difficult for measures of changes in
student learning
– Little research on institutional change
Stanford Faculty Development Center
Faculty Development Outcomes


Are the data hidden in the process we are
observing?
• Curricular Change?
• Emphasis on Teaching?
• New Teaching Methods?
• New Observations of Faculty Needs?
• Requirements for Teaching Improvement
Methods by Accrediting Agencies?
Areas for Further Work
• Further Enhancement of Pedagogy –
Improve the best
• Pedagogical Content Knowledge (Shulman)
• Needs of the Profession
Types of Faculty Development
Areas for Further Work




Instructional development
Professional development
Leadership development
Organizational development
Irby, 1995, from Berquist and Phillips
Types of Faculty Development
Your Beliefs
Instructional development
 Should be improved?
 Can be improved?
 Has been improved?
Professional development
 Should be improved?
 Can be improved?
 Has been improved?
Leadership development
 Should be improved?
 Can be improved?
 Has been improved?
Organizational development
 Should be improved?
 Can be improved?
 Has been improved?
Lessons from Faculty
Development in Clinical Teaching

Current Challenges/Opportunities
• INSTITUTIONAL
–
–
–
–
–
Other Alligators
Attitude - good enough mentality?
TIME / MONEY/ ADMINISTRATIVE SUPPORT?
EVIDENCE FOR NEED: Present, ? compelling
WHO IS GOING TO PAY?
• TEACHER’S THEMSELVES
– Philosophical/practical/self-assessment/
• METHODS: Applicabililty to current
environment
Stanford Faculty Development Center
Areas for Increased Emphasis in
Faculty Development

INSTITUTIONAL BENEFITS
– IDENTIFICATION OF NEW AREAS FOR
IMPROVEMENT - Consolidation seminar - ->>
• STRUCTURAL RECOMMENDATIONS
• APPLICATION OF KNOWLEDGE
RECOMMENDATIONS FOR INSTITUTIONS, E.G.,
MDM POLICIES, PATIENT CARE
• EDUCATIONAL RECOMMENDATIONS, E.G.
TEACHING RESOURCES, ADMINISTRATIVE
RESOURCES

SOCIETAL BENEFITS
– IMPROVED OUTCOME FOR LEARNERS -->>
FUTURE PHYSICIANS/SCIENTISTS
Stanford Faculty Development Center
Lessons from Faculty Development in Clinical
Teaching
Data vs Belief
Importance of Faculty Development
 Needs of Educators Academic / Community
 Topics of interest and importance
(Content and process)
 Benefits of Development Methods
 Current Challenges and Opportunities

Stanford Faculty Development Program