SoTL 1
Introduction to the Scholarship of Teaching & Learning
• Welcome!
• Please sit in groups of four and introduce
yourselves to those you don’t know.
• The manager is the person who has traveled
the farthest to come to the meeting.
• The manager should appoint a recorder and
reporter.
Workshop Goals
During this workshop, each participant will:
• Develop a working definition of the scholarship
of teaching and learning (SoTL)
• Articulate a SoTL question that can be
addressed in one’s classroom
• Identify the basic components of a SoTL
project and resources that can be used to move
a project forward
Relevant Terminology
• Individually, on your index card, define the
following (2 min):
• Scholarship of Teaching and Learning (SoTL)
• Assessment
• Evaluation
• In your group, compare definitions (4 min).
• Be prepared to report on how much agreement or
disagreement your group had in the definitions.
POGIL’s Working Definitions: SoTL
• Various definitions exist in the literature and in
practice.
• POGIL’s working definition of SoTL:
Investigations that begin with the purpose of
understanding and improving teaching and learning
within a classroom or institution, leading to results
that can be shared beyond your classroom.
The Ongoing Cycle of Scholarly Teaching & the
Scholarship of Teaching and Learning (SoTL)
Knowledge
Base
about
Teaching
and
Reflection on
Teaching/Learning
Modify
Pedagogy
Scholarly
Teaching
Assess
Success
Scholarship
of Teaching
Share results
Presentation
Peer Review
Learning
Publication
Gurung & Schwartz (2009) adapted from Richlin, © 1993
Frameworks for Inquiry
Consider the cookies
• What kind of questions might you ask about
these cookies?
• Feel free to Explore the cookies and as a
group write one question you might ask about
the cookies.
Assessment vs. Evaluation
• Assessment is analysis and use of data to make
improvements in performance.
• Evaluation is analysis and use of data to make
judgments about performance.
http://global.cscc.edu/assessment/AssessEval.shtml
Assessment vs. Evaluation: Details
Topic
Assessment
Evaluation
Cookies
• Establish optimal characteristics
• Good cookie
• Chewy
• Satisfactory cookie
• Crunchy
• Unsatisfactory cookie
• Sweetness level
• Composition, etc.
Free
throw
shots
• Analyze characteristics of success
• Coach to improve characteristics of
success
• Percentage of successful
shots has reached an
acceptable level (or not)
Assessment vs. Evaluation
Writing Assignments
Assessment is analysis and
use of data to make
improvements in
performance.
Evaluation is analysis and
use of data to make
judgments about
performance.
Comments on the first draft
allow students to make
changes to improve their
work.
Grade on final draft is
overall judgment of quality
of written assignment.
FORMATIVE
SUMMATIVE
SoTL Questions
• SoTL inquiry into classroom teaching
• Focuses on gathering data that can be used to
improve
• (Formative) Assessment data!
• Need to start with a good SoTL question
Foci of Assessment
Academic Focus
Situational Focus
• Content Knowledge
• Classroom Activity
• Process Skills
• Classroom Facilitation
• Content Unit
• Entire Course
• Program Curriculum
Types of SoTL questions
• Idea development
ideas
• Range of
understanding
• Long-term learning
• Depth of
understanding
• Measure of critical
thinking
• Misconceptions
• Understanding difficult
• Transfer of learning
• Progressive dialogue
• Instructional practice
Savory, P., Burnett, A.N., & Goodburn, A. (2007). Inquiry into the College Classroom:
A Journey Toward Scholarly Teaching. Boston, MA: Anker. p. 10
Let’s Get Started
What Is Your Question?
• Find an issue that is important to you.
• My students struggle with graphs
• My students don’t seem to get much out of the
assigned reading
• Write a question you have about it.
• What activities help students scaffold their mastery
of reading graphs?
• What are my students doing when they read
the textbook?
What Is Your Question?
1. Use the index card to record a question that would
be of interest for you to investigate in your
classroom (2 min).
2. Categorize your question in at least one of the
following ways (1 min).
a. Identify its academic focus (content or process skill).
b. Identify its situational focus (a single assignment,
a content unit, the entire course, etc.)
c. Identify what type of inquiry it is
(idea development, etc.)
Choose a Question To Work
on in Your Group
In Groups (4 min):
• Share your questions.
• Select one question in your group to explore
in depth.
• Write it at the top of your large post-it note.
(Leave room for revisions of the question.)
Characteristics of a Good Question
• Focused.
• Specific.
• Measurable.
• Variables can be identified.
Characterizing Questions
• Specific: terms mean a
particular thing
• Focused: scope is one that you
can answer within your own
teaching context
• Measurable: provides a strategy
for collecting data
• Variables can be identified
many (all) of potential
influences can be identified.
• Is POGIL better than lecture?
• Is my implementation of POGIL
more effective than my
implementation of lecture?
• Does my implementation of
POGIL result in improved
student understanding of how
to analyze a graph, compared to
my implementation of lecture?
• If I add activities which
specifically address or utilize
the interpretation of graphs,
does student understanding
improve?
Revise your question
In Groups (5 min):
• Consider how you might refine your question to
make it more:
• Specific.
• Focused.
• Measureable.
• Variables can be identified.
• Recorder: write the revised version of the question
on your group’s large post-it.
Planning to Collect Data
What is Your Investigative Plan?
Consider:
• Qualitative data (results can differ with observer)
• e.g., student responses scored by rubrics, interview
data
• Quantitative data (constant across observers)
• e.g., test scores, percentage of students in particular
demographic categories, likert-scale survey data
• Both measures are equally valid, but different.
Planning to Collect Data
What is Your Investigative Plan?
Likert scale surveys provide quantitative data
from qualitative data. Example:
When I read the textbook, I am able to identify and
understand the major concepts/ideas in each chapter.
a. Strongly agree
b. Agree
c. Neither agree nor disagree
d. Disagree
e. Strongly disagree
Planning to Collect Data
What is Your Investigative Plan?
Take two minutes in your group to come up
with some types of evidence for each column.
Qualitative
Quantitative
Types of Classroom Data
Grades are not the only source of data in
a classroom:
• Assignments can be designed to help students
learn and provide data for classroom
investigations.
• The use of grading rubrics allows for
comparison of student work within or across
semesters.
Tools for SoTL Data collection
Types
Where to find them
• Surveys
• Rubrics
• Books on classroom
assessment
• Interviews
• Literature
• Achievement questions
• Professional societies
• Standardized exams
• Colleagues in pertinent
disciplines
• Institutional research
office
When Thinking About Instruments
• Validity: The tool
measures what you think
it measures.
• You don’t measure the
temperature with a
measuring spoon.
• Reliability: The tool will
give you consistent
answers.
• When you set the oven for
400ºF, does it always
reach 400ºF?
Caveats
• Survey questions can be
challenging to write
• Using a ready-made
instrument that was not
designed for your purpose
might lead to data that are
not meaningful or that are
misleading.
• Modifying an instrument
affects its reliability and
validity.
Data Analysis
• Go beyond raw data:
• Percentages
• Normalization
• Graphs
• Common themes
• Carry out statistical tests:
• Are the differences significant?
• Effect size
Refine Your Investigative Method
As a group, come up with a plan to answer your
question (8 min).
• What measurements will allow you to distinguish among
different outcomes?
• Match the methodology and tool to the question being
asked.
• Quantitative/Qualitative/Mixed methods.
• Triangulation: Collection of data from different sources to
measure a single effect that cannot be measured directly.
If you end up revising your question, record the new version
on your large post-it.
Insights
In your group, discuss what insights your group
has had regarding the process of designing a
SOTL project. Make sure a group member is
designated to report out. (4 min)
Things To Do To Be Ready for a SoTL Project
• Think small/modest (at least at the beginning)
• Save everything!
• Notes on what you did in class and reflections on how it went
• All student work (scan exams, homework, lab notebooks, etc.)
• Periodic student feedback or self-assessments
• Investigate your institution’s rules on gathering data
from/about students. Get IRB (Institutional Research
Board) approval, so that any data you collect can be shared
beyond your own classroom.