Gathering Data for
Faculty Evaluation
Shelley A. Chapman, PhD
Senior Educational Consultant,
The IDEA Center
Plan for this Workshop
I. Foundations to Gathering Data for
Faculty Evaluation
II. Logistics of Gathering Data for Faculty
Evaluation
III. Practicing Classroom Visitations and
Observations
I. Foundations
Foundations to Gathering Data for
Faculty Evaluation
• Philosophical Underpinnings
• Administrative Foundations
• Classroom Observations as Ethnographic
Practice
• Classroom Visitations as the Community of
Practice
Philosophical Underpinnings
Learning Tree
Skills
Knowledge
Emotions
Assumptions, Beliefs, Values
© Shelley A. Chapman 2007
Learning About Faculty Evaluation
Can faculty and administrator engage in
a holistic system of data collection and
Interpretation?
Skills
Knowledge
Do faculty and administrator what the system is,
which data are collected, how data are used?
Emotions How do faculty and administrators feel about the
System? Are the results being used in a fair and
appropriate way?
Assumptions, Beliefs, Values
What might be some assumptions,
beliefs, and/or values about what a good faculty
evaluation system looks like?
A. Multiple forms of Assessment
Student
Ratings
Artifacts
External
Perspective
Balanced
Plan for
Summative
Evaluation
Evidence of Good Teaching
Student
Ratings
• Administer Appropriately
• Collect 6-8 Reports
(more if class size is <10)
• 30-50% of Overall Evaluation
Evidence of Good Teaching
Student
Ratings
• Student Comments-formative
• Be mindful of standard error
of measurement (±.3)
• Use 3-5 Performance
Categories
Evidence of Good Teaching
Artifacts
• Syllabi
• Graphic Organizers
• Assignments and project
descriptions
• Rubrics
• Written Teaching
Philosophy/Reflections
• Samples of Student Work
• CATs and results
Evidence of Good Teaching
External
Perspective
• Focus Groups of Graduating
Students
• Alumni Surveys
• Invited Presentations
• Classroom Observation
• Classroom Visitation
B. Qualitative Research
• Analyzing at a phenomenon or situation
• Questions: Why? How? In what way?
• Goal  Understanding of human or social
phenomenon
• Tends to be inductive
Overarching Goal
Critical Reflection
Try
Something
New
Feedback
Analysis
Administrative Foundations
• Systematic and Thoughtful
• Formative and Summative
• Multi-dimensional and use multiple
sources
• Clear Guidelines should be established*
*Adapted from Van Note Chism, H. (2007). Peer review of teaching: A
sourcebook. Bolton, MA: Anker.
The Evaluation Program:
• Link with faculty rewards and awards
• Present in a straighforward, accurate, and
complete way
• Focus on teaching improvement
• Use valid and reliable instruments
• Requires professional judgment to make
subjective decisions (even with objective
data)
Three-Phase Process for
Faculty Evaluation
Use
Collect
Data
Set
Expectations
Data
Create Categories of Performance
Below
Acceptable
Standards
Marginal,
Needs
Improvement
Does Not
Meet Expectations
Meets
Expectations
Meets Expectations
Exceeds
Expectations
Outstanding
Exceeds Expectations
Be Clear
• Purpose?
• Summative and formative?
• Who or what is being reviewed?
• Outcomes? What if it is strongly negative?
• Is the person, unit, or program undergoing
the review aware of the possible
outcomes?
Be Clear (Continued)
• With whom will the results be shared?
• What is the nature of my conclusion? Am I
authorized to make a decision or am I making
a recommendation to a chair or dean?
• Should I retain copies of the records?*
*Adapted from
Buller, J.L. (2012). Best practices in faculty evaluation: A practical guide for academic leaders.
San Francisco: Jossey-Bass.
Classroom Observation
• Deductive Approach
• See Saint Francis University’s
• “Class Visitation Form”
• Classroom Observation Form Open Ended
(Form A)
• Framework of Faculty Teaching Performance
Evaluation
Classroom Observation
• Inductive Approach
Classroom Observation
as
Ethnographic Practice
Ethnographic Practice
The Process is
Observation in the “field”
to understand the context…
Characterized by “thick” description
using the lens of culture
Ethnographic Practice
The Product is
A report that includes rich descriptions…
themes/categories...
trends…
and some interpretation
Classroom Observation
• Why?
To understand the context of a particular
classroom.
• Who?
Trained observer and any class
• When
Mid-way through the course or so
• What
“Observation” in the ethnographic sense
• How
As an ethnographer would take field notes
Classroom Visitation
What is the value of peer learning?
Gestalt: The whole is greater than the sum
of its parts.
Faculty learn from their colleagues as they
form a “Community of Practice.”
Research on Peer Learning
“…peer learning among small groups of
teachers was the most powerful
predictor of improved student
achievement over time.”
Quoted by
Darling-Hammond, L. (2013). Getting teacher evaluation right: What
really matters for effectiveness and improvement. Teachers College
Press: NY.
What is a Community of Practice?
Group of people who
share a concern or a
passion for something
they do and learn how to
do it better as they
interact regularly.
Resource:
Wenger, E. , McDermott, R., &
Snyder, W. (2002). Cultivating
Communities of Practice: A guide
to managing knowledge.
Cambridge, MA: Harvard
Business Press.
Communities of Practice
• New Faculty/Adjunct Faculty
• Tenure Track Faculty
• Specific type of format
• Flipped
• Hybrid
• Labs or clinicals
• Large lecture
Communities of Practice
• Specific type of technique
• Problem-based Learning
• Case study method
• Jigsaw
• Chalk Talk
• Think-share-pair
Communities of Practice
• Specific type technology
• Clickers
• Smartboards
• Video clips
• Other technology
Communities of Practice
• Specific type of issues
• Discipline problems
• Motivation problems
• Difficult concepts
• First-year students/first generation
college students
• Laptops and phones in the classroom
Examples
• Amanda in Biology for Non-majors
• Goal: Improve student understanding of biological concepts
• Help: Tom uses clicker feedback technology for his large political
science course.
Tom’s Political Science Class
1. How similar is Tom’s class to mine (in demographics)?
2. What type of subject matter is Tom addressing (factual, quantitative,
conceptual, theoretical, procedural, etc.)
3. What was his first question for which he used clickers?
4. How did student responses help Tom to inform or adjust his teaching?
5. How often did Tom use clicker feedback throughout the class?
6. What problems occurred with the technology or process?
7. What can I learn from this class that might I adapt and use in my class?
Examples
• Jeff is teaching a capstone class in the Business/Management
major
• Goal: Help student integrate theory and practice in real life situations
• Help: Susan uses the case study method in her Marketing class
Susan’s Marketing Class
1. How similar is Susan’s class to mine (in demographics)?
2. What types of learning does Susan want her students to apply? Principles?
Skills?
3. When did the students receive the case (before or in class)? How long is it?
4. What was Susan’s role with the case?
5. What did students do with the case? Did they work in groups? What was
the desired outcome? Presentation? Report?
6. What problems occurred with the process?
7. What can I learn from this class that might I adapt and use in my class?
Examples
• Jan is a new faculty member, teaching Math in her first semester
• Goal: Help students to be engaged and on task
• Juan has been teaching Math successfully at this institution for 8 years
Juan’s Class
1. How similar is Juan’s class to mine (in demographics)?
2. What types of learning does Juan want his students to apply? Principles?
Skills? Both?
3. How does Juan handle homework? Do students hand it in? Do they use it in
class?
4. What was Juan’s role with the homework? Is it graded?
5. Are students distracted by mobile phones and laptops? If so, how does Juan
handle that?
6. Were students engaged? If so, why? If not, why not?
7. What can I learn from this class that might I adapt and use in my class?
II. Logistics: Visitation (Peer)
Establish
Purpose
and
protocol
Identify
CoPs
Select
Tools/
Reflective
Questions
Invite
Participants
Prepare
Participants
7 Principles for Cultivating a CoP*
1. Design for Evolution
2. Open a Dialogue Between Inside and
Outside Perspectives
3. Invite Different Levels of Participation
4. Develop both Public and Private
Community Spaces
5. Focus on Value
6. Combine Familiarity and Excitement
7. Create Rhythm for the Community
*Wenger, E., McDermott, R., Snyder, W.M. (2002). Cultivating communities of practice:
A guide to managing knowledge. Cambridge, MA: Harvard Business Press.
Individualizing Visitations
Identify
need
Match up
peer
Meet
with peer
to discuss
need
Design or
select
Guiding
Questions
and
Review
course
materials
Visit Class
(host
faculty
introduce
visitor)
Write
short
reflection
/ Debrief
Visitation Logistics
• Use for formative purposes (usually for the
observer, but can be reciprocal)
• Visit someone’s class before having
someone visit yours
• Make sure it is a good match—it will meet
your specific needs
• Keep it low stakes and relevant
Before the Visit
• Ask if the host faculty has a teaching
philosophy to share
• Review the syllabus, where the class is in
sequence
• Review any other course materials
pertinent to the visit
• Ask for any insights about the class that
the instructor may want to share
During the Visit
• Sit in the back of the room
• Allow yourself to be introduced
• Use a set of focused questions
• Take class materials (text, syllabus,
etc)
• Observe and take notes, answering
your questions
After the Visit
• Write a brief reflective statement
• Meet with the instructor to debrief
• Plan next steps for your class
• Invite faculty member to visit as
appropriate
II. Logistics:
Observations
• Observer should be trained and should
practice
• Results can be Formative and Summative
• Meet with the faculty being observed
before and after the observation
• Prepare materials to record observations
Before the Observation
• Ask if the host faculty has a teaching
philosophy to share
• Ask for any insights about the class that
the instructor may want to share
• Review the syllabus, where the class is in
sequence
• Review any other course materials
pertinent to the visit
Review of Materials
• Formative Purposes:
• Include discussions of materials in department
meetings
• Encourage a “sharing of materials” session
over refreshments
• Establish protocols for standard syllabus
template
• Provide examples of well-designed
assignments
• Effective and Engaging
• Provide examples of rubrics
Review of Materials
• Summative Purposes:
• Create templates and use them to judge the
quality of faculty developed products
• Create rubrics to use to judge the quality of
assignments or assessments
Classroom Observation Log
Time What Happened
What Was Said
Classroom Observation Log
Time
What Happened
What Was Said
8:05
Instructor shut door
Instructor (I): OK, Class. Let’s begin.
Make sure you turned in your
homework as you came in.
Students are shuffling papers,
opening books.
Today we will begin our discussion
on the brain. Turn in your textbooks
to chapter 5.
8:10
Student comes in late
Several students raise hands
8:15
Female in first row is called on
Is your brain more like a computer
or a jungle? Who would like to
respond first?
Student (S) My brain is a jungle! I am
so unorganized! (class laughs)…
Flow of Communication Map
Instructor
M
M
F
F
F
F
F
M
M
M
M
M
F
F
M
M
F
F
F
F
F
Instructor
F
M
M M
M
F
M
M
M
M
M
M
M
M
M
M
F
M
M
M
M
F
M
Examples
• Business Class with very few women
• Business Class with students from diverse
backgrounds
• Accounting Class with adjunct instructor who
was accused of racial discrimination
• English instructor who was notorious for being
unprepared
During the Observation
• You will be busy!
• You won’t get everything!
• You are capturing all you can.
• Try to keep up for about 45-50
minutes
After the Observation
• Review notes for general impressions
• Type up report
• Share report with instructor for feedback
• Allow instructor to submit written
feedback if desired
• Follow protocol as to who will receive
copies of the report
• Remember that all data are confidential
See Sample Reports
• Professor Jones
• Professor Smith
Thank you.
Questions?
Shelley@theideacenter.org