Task
1. How would you describe your experience
with Scholarship of Teaching and Learning?
Circle one: Extensive Moderate Hardly-Any None
2. OUTCOMES: List 3 topics or issues you
hope to learn more about at this workshop
.
Continuum of Experience with SOTL
A Metaperspective
Α
Ω
Origin of SOTL
What & Why of SOTL
Initiating a SOTL
Program
Faculty SOTL
Projects
Bridges to Productivity
Questions, Designs &
Resources
A Conceptualization of Teaching Related Activities
Teaching
Scholarly
Teaching
Scholarship of
Teaching &
Learning
Scholarly Teaching:
Teaching that entails certain practices of classroom assessment and
evidence gathering; teaching that is informed not only by the latest
ideas in the field but by current ideas about teaching generally and
specifically in the field; and teaching that invites peer collaboration or
review.
Scholarship of Teaching:
An act of intelligence or artistic creation becomes scholarship when it
possesses at least three attributes: it becomes public, it becomes an
object of critical review and evaluation by members of one’s
community, and members of one’s community begin to use, build
upon, and develop those acts of mind and creation.
(Definitions after Shulman)
An ineffective
deed of teaching
An excellent deed of
teaching
A superior deed
of scholarly
teaching
Quality (or “excellence”) of any teaching-related
activity is an independent dimension not represented
in the plane of the Venn diagram.
Teaching
Scholarly
teaching
Scholarship
Contemporary forms of
research and creative activity
Distinction between scholarship of teaching and learning
as a domain of academic achievement and as a campus
program:
SOTL as academic achievement (yellow ellipse)
SOTL as a campus program can span all teaching
related activities (leftmost three ellipses).
A SOTL program encourages movement in teachingrelated activities toward the right in the diagram.
Why SOTL?
•
•
•
•
•
Synthesis
Building community
Wisdom of practice
Generativity
Reserve capacity
Task
• What reasons, these or others, are most
important on your campus?
• What reasons might be irrelevant to faculty
on your campus?
Origin and Evolution of SOTL (In Progress)
Not a new idea (Hutchins 1923)
“Scholarship of teaching” coined (Boyer 1990)
Reform Concepts
Implementing Strategies Recent Articulations
CA/CR (Angelo &
Cross 1993)
AAHE FFRR
New epistemology
(Schön 1995)
- Carnegie Scholars
New American scholar
(Rice 1996)
- Professional Societies Kreber & Cranton
JHE 71 (4) 2000
- Knowledge Media Lab
Richlin in Kreber (Ed.)
Peer Review of Teaching
New Directions 86 200
Scholarship Assessed
(Glassick et al 1997)
CASTL
- Teaching Academy
Shulman, Change
31 (4) 1999
Hutchings & Shulman
Change 31 (5) 1999
What is certain is that most Ph.D.’s become
teachers and not productive scholars as well. [A
Ph.D. candidate who plans to be a teacher] must
know his field and its relation to the whole body
of knowledge. It means too that he must be in
touch with the most recent and most successful
movements in undergraduate education, of which
he now learns officially little or nothing. How
should he learn about them? Not in my opinion by
doing practice teaching upon the helpless
undergraduate. Rather he should learn about them
through seeing experiments carried on in
undergraduate work by the members of the
department in which he is studying for the degree”
Robert Maynard Hutchins
Origin and Evolution of SOTL (In Progress)
Not a new idea (Hutchins 1923)
“Scholarship of teaching” coined (Boyer 1990)
Reform Concepts
Implementing Strategies Recent Articulations
CA/CR (Angelo &
Cross 1993)
AAHE FFRR
New epistemology
(Schön 1995)
- Carnegie Scholars
New American scholar
(Rice 1996)
- Professional Societies Kreber & Cranton
JHE 71 (4) 2000
- Knowledge Media Lab
Richlin in Kreber (Ed.)
Peer Review of Teaching New Directions 86 200
Scholarship Assessed
(Glassick et al 1997)
CASTL
- Teaching Academy
Shulman, Change
31 (4) 1999
Hutchings & Shulman
Change 31 (5) 1999
“On the high ground, manageable problems lend
themselves to solution through the use of researchbased theory and technique. In the swampy lowlands,
problems are messy and confusing and incapable of
technical solution. ...the problems of the high ground
tend to be relatively unimportant to individuals or to
the society at large, however great their technical
interest may be, while in the swamp lie the greatest
problems of human concern.
...Shall [the
practitioner] remain on the high ground where he can
solve relatively unimportant problems according to
his standards of rigor, or shall he descend to the
swamp of important problems where he cannot be
rigorous in any way he knows how to describe?”
CONCRETE
CONNECTED
KNOWING
ACTIVE
PRACTICE
REFLECTIVE
OBSERVATION
ABSTRACT
ANALYTIC
KNOWING
R. Eugene Rice. 1996. The New American Scholar. AAHE. p. 14
Origin and Evolution of SOTL (In Progress)
Not a new idea (Hutchins 1923)
“Scholarship of teaching” coined (Boyer 1990)
Reform Concepts
Implementing Strategies Recent Articulations
CA/CR (Angelo &
Cross 1993)
AAHE FFRR
New epistemology
(Schön 1995)
- Carnegie Scholars
New American scholar
(Rice 1996)
- Professional Societies Kreber & Cranton
JHE 71 (4) 2000
- Knowledge Media Lab
Richlin in Kreber (Ed.)
Peer Review of Teaching
New Directions 86 200
Scholarship Assessed
(Glassick et al 1997)
CASTL
- Teaching Academy
Shulman, Change
31 (4) 1999
Hutchings & Shulman
Change 31 (5) 1999
Origin and Evolution of SOTL (In Progress)
Not a new idea (Hutchins 1923)
“Scholarship of teaching” coined (Boyer 1990)
Reform Concepts
Implementing Strategies Recent Articulations
CA/CR (Angelo &
Cross 1993)
AAHE FFRR
New epistemology
(Schön 1995)
- Carnegie Scholars
New American scholar
(Rice 1996)
- Professional Societies Kreber & Cranton
JHE 71 (4) 2000
- Knowledge Media Lab
Richlin in Kreber (Ed.)
Peer Review of Teaching New Directions 86 200
Scholarship Assessed
(Glassick et al 1997)
CASTL
- Teaching Academy
Shulman, Change
31 (4) 1999
Hutchings & Shulman
Change 31 (5) 1999
CASTL = Carnegie Academy for the
Scholarship of Teaching & learning
• Higher Education
• K-12
CASTL = Carnegie Academy for the
Scholarship of Teaching & learning
•
•
•
•
Carnegie Scholars
Teaching Academy
Professional Societies
Related Carnegie Initiatives
– Knowledge Media Lab
– Resources
http://www.carnegiefoundation.org/CASTL/highered/index.htm
CASTL = Carnegie Academy for the
Scholarship of Teaching & learning
• Pew National Program for Carnegie
Scholars
http://www.carnegiefoundation.org/CASTL
/highered/Pewscholars.htm
CASTL = Carnegie Academy for the
Scholarship of Teaching & learning
• Teaching Academy Campus Program
AAHE & Carnegie
– Level I: Campus Conversations
– Level II: Going Public
– Level III: National Teaching Academy
http://www.carnegiefoundation.org/CASTL/highered/teachingacademy.htm
CASTL = Carnegie Academy for the
Scholarship of Teaching & learning
• AAHE CASTL Campus Program
WebCenter
– Campus Reports
– List of "Going Public Grants"
– Resources
http://aahe.ital.utexas.edu
CASTL = Carnegie Academy for the
Scholarship of Teaching & learning
• Professional Societies
– Networking
– Invitational Small-Grants
– Register & Reports
http://www.carnegiefoundation.org/CASTL/highered
/collaborations.htm
CASTL = Carnegie Academy for the
Scholarship of Teaching & learning
• Knowledge Media Lab
– Tour
– Exhibition: Scholarship Under
Construction
– Gallery
– Multimedia Tutorial for SOTL
http://www.carnegiefoundation.org/KML/index.htm
CASTL = Carnegie Academy for the
Scholarship of Teaching & learning
• [Other] Resources
– Bibliography of SOTL
– Text of Key Papers
Origin and Evolution of SOTL (In Progress)
Not a new idea (Hutchins 1923)
“Scholarship of teaching” coined (Boyer 1990)
Reform Concepts
Implementing Strategies Recent Articulations
CA/CR (Angelo &
Cross 1993)
AAHE FFRR
New epistemology
(Schön 1995)
- Carnegie Scholars
New American scholar
(Rice 1996)
- Professional Societies Kreber & Cranton
JHE 71 (4) 2000
- Knowledge Media Lab
Richlin in Kreber (Ed.)
Peer Review of Teaching New Directions 86 200
Scholarship Assessed
(Glassick et al 1997)
CASTL
- Teaching Academy
Shulman, Change
31 (4) 1999
Hutchings & Shulman
Change 31 (5) 1999
An Exemplary Course Portfolio
And
A Superb Model of SOTL
An Alternative Approach to General
Chemistry
Assessing the Needs of ‘At-Risk’ Students with Cooperative
Learning Strategies
Dennis Jacobs
Professor of Chemistry
University of Notre Dame
Rita Naremore, Simon Brassell, Shanker Krishnan, David Parkhurst
Constructing and Evaluating a Course Portfolio:
Making Good Teaching Apparent
October 26 2000
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies
Dennis Jacobs (University of Notre Dame)
Contents of Course Portfolio:
QuickTim e™ and a
GIF decom pressor
are needed to see this picture.
• Overview
• Rationale:
• Why develop an alternative approach?
• Implementation
• Changes introduced to foster learning.
• Targeting ‘at-risk’ students.
• Impact
• Documentation and assessment of immediate and longerterm effects.
• Library
• Examples of videos of group discussions, tests, on-line
quizzes, questionnaires.
http://kml.carnegiefoundation.org/gallery/djacobs/
-2-
Constructing and Evaluating a Course Portfolio:
Making Good Teaching Apparent
October 26 2000
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies
Dennis Jacobs (University of Notre Dame)
QuickTim e™ and a
GIF decom pressor
are needed to see this picture.
Course Portfolio Design:
Constructing and Evaluating a Course Portfolio:
Making Good Teaching Apparent
October 26 2000
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies
Dennis Jacobs (University of Notre Dame)
QuickTim e™ and a
GIF decom pressor
are needed to see this picture.
Rationale:
• Recognized Problems:
• ‘At-risk’ students (Math SAT ≤630):
• dropped out of General Chemistry.
• didn’t take any advanced science.
• frustrated by large lecture format.
• Alternate Course Design:
• Similar requirements and lectures.
• Comparable exams.
• Various activities involving structured cooperative learning.
• Initial Comments on Proposal
• “Only delaying inevitable failure.”
• “Efforts should be focused on the best not the ‘at-risk’ students.”
-4-
Constructing and Evaluating a Course Portfolio:
Making Good Teaching Apparent
October 26 2000
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies
Dennis Jacobs (University of Notre Dame)
QuickTim e™ and a
GIF decom pressor
are needed to see this picture.
Documentation:
• ‘At-risk’ students
markedly more likely to
drop or fail the course
in the traditional class
format.
-5-
Constructing and Evaluating a Course Portfolio:
Making Good Teaching Apparent
October 26 2000
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies
Dennis Jacobs (University of Notre Dame)
Implementation:
QuickTim e™ and a
GIF decom pressor
are needed to see this picture.
• Aim:
• Provide improved learning opportunities for ‘at-risk’
students.
• Develop more effective teaching in large lecture format.
• Alternate Course Design:
• Introduced opportunities for structured cooperative
learning including:
• discussion of concepts in pairs.
• small group in recitation sections.
• work as pairs in laboratory.
• Mandatory recitation sections:
• more time committed to class.
• direct contact with instructors.
-6-
Constructing and Evaluating a Course Portfolio:
Making Good Teaching Apparent
October 26 2000
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies
Dennis Jacobs (University of Notre Dame)
QuickTim e™ and a
GIF decom pressor
are needed to see this picture.
Traditional vs. Alternative Classes:
• Similarities:
•
•
•
•
Size (250 students), text, chapters.
Lecture time (3 hr), lab time (2.5 hr).
Lecture format (Powerpoint slides and demonstrations).
Exam format and many exam questions.
• Differences in Alternative Section:
• Mandatory recitations (1 hr/wk, 20 students); attendance 95%
vs. 10%.
• Weekly homework (10 vs. 30, graded).
• On-line quizzes (www chapter reviews).
• Weekly feedback from homework, group problems, on-line quiz,
in-class questions.
• Personal contact with instructor and follow-up if performance
declined.
-7-
Constructing and Evaluating a Course Portfolio:
Making Good Teaching Apparent
October 26 2000
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies
Dennis Jacobs (University of Notre Dame)
QuickTim e™ and a
GIF decom pressor
are needed to see this picture.
Impact of Alternative Section:
• Assessment Strategies:
• Effects on conceptual understanding, problem-solving and selfconfidence:
• feedback from students.
• evaluation of individual elements of the cooperative
learning activities.
• Immediate and long-term benefits:
• retention of ‘at-risk’ students.
• success in advanced science classes.
• Data Collection:
• Recording in-class learning activities.
• Tracking individual grades and progress.
• Longitudinal study of ‘at-risk’ students:
• progress in subsequent classes.
-8-
Constructing and Evaluating a Course Portfolio:
Making Good Teaching Apparent
October 26 2000
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies
Dennis Jacobs (University of Notre Dame)
Measurement of Impact:
QuickTim e™ and a
GIF decom pressor
are needed to see this picture.
• Success of ‘At-risk’ Students:
• Better grades in General Chemistry.
• Improved retention in class.
• Higher success rate in subsequent classes.
-9-
Constructing and Evaluating a Course Portfolio:
Making Good Teaching Apparent
October 26 2000
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies
Dennis Jacobs (University of Notre Dame)
QuickTim e™ and a
GIF decom pressor
are needed to see this picture.
Measurement of Impact — II:
- 10 -
Constructing and Evaluating a Course Portfolio:
Making Good Teaching Apparent
October 26 2000
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies
Dennis Jacobs (University of Notre Dame)
QuickTim e™ and a
GIF decom pressor
are needed to see this picture.
Measurement of Impact — III:
- 11 -
Constructing and Evaluating a Course Portfolio:
Making Good Teaching Apparent
October 26 2000
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies
Dennis Jacobs (University of Notre Dame)
QuickTim e™ and a
GIF decom pressor
are needed to see this picture.
Conclusions:
• Dennis Jacobs’ Course Portfolio:
• Succinct and reflective.
• Focused on specific learning objectives.
• Uses readily available information:
• grades, course materials, etc.
• Documents the positive effects of changes in instructional
methodology.
• Benefit for Dennis Jacobs:
• Ready justification for resources to support recitation
sections.
• Value of Course Portfolios:
• Demonstrable outcomes of teaching practices that can be
peer-reviewed.
- 12 -
Constructing and Evaluating a Course Portfolio:
Making Good Teaching Apparent
October 26 2000
Origin and Evolution of SOTL (In Progress)
Not a new idea (Hutchins 1923)
“Scholarship of teaching” coined (Boyer 1990)
Reform Concepts
Implementing Strategies Recent Articulations
CA/CR (Angelo &
Cross 1993)
AAHE FFRR
New epistemology
(Schön 1995)
- Carnegie Scholars
New American scholar
(Rice 1996)
- Professional Societies Kreber & Cranton
JHE 71 (4) 2000
- Knowledge Media Lab
Richlin in Kreber (Ed.)
Peer Review of Teaching New Directions 86 200
Scholarship Assessed
(Glassick et al 1997)
CASTL
- Teaching Academy
Shulman, Change
31 (4) 1999
Hutchings & Shulman
Change 31 (5) 1999
Task
Think about your own background in
SOTL. What additions would you make
to the map “Origins and Evolution of
SOTL” in each category?
Reform Concepts
Implementing Strategies
Recent Articulations
Outline
The IUB Experience
• The Administrative Perspective
– Design Principles
– Kinds of Resources Needed
– Changes to the Faculty Culture
• The Departmental Perspective
– Going from Zero to Sixty in 10.2 Seconds
• The Individual Faculty Member’s Perspective
How Did We Set Up a Campus
SOTL Program?
The Ingredients:
– A dedicated director
– A faculty advisory committee
– A core of committed researchers
willing to share their work
– Massive Administrative Support
The Key: Faculty Input
Design Principles
• Get out the crowd to initial events
• Serve food
• Connect with national
– Programs
– Resources
• Be inclusive
Design Principles (Cont.)
• Involve librarians
• Partner with stakeholders
• Identify needs
– Campus conversations
– Faculty committees
• Exhibit administrative support
What Resources Were Needed?
• Financial—Money is a great motivator!
• Human—People will go where the money is!
• Structural—Other rewards are important in
the academic environment. Integrate SOTL
into this reward structure.
Deploying Available Resources: Some
Examples
•
•
•
•
Expand rewards
Travel grants
Presentation/research grants
Visibility for contributions
ONE EXAMPLE OF A STRUCTURAL CHANGE
IU FACULTY SUMMARY REPORT PRIOR TO ACADEMIC
YEAR 1999-2000
TEACHING ACTIVITIES
A. Courses taught (weekly contact hours reported by course
number in tabular form).
C. Development or major revision of course(s) during the year.
D. Dissertation, Research and Field Work Committees
E. Teaching awards and honors, including those of your students.
FACULTY SUMMARY REPORT
[Revised in Academic Year 1999-2000]
TEACHING ACTIVITIES
A. Courses taught (weekly contact hours reported by course
number in tabular form).
B. Activities directed at improving instruction, learning, or
course administration. (Please describe rationale
for/description of innovations, methods/measures for
assessing outcomes, and results.)
Please note: Scholarly activity related to teaching and
learning (e.g. investigation/research,
dissemination/publication of results) should be reported
under the section on Research/Creative Activities.
FACULTY SUMMARY REPORT,
CONTINUED
C. Development or major revision of course(s) during
the year.
D. Dissertation, Research and Field Work Committees
E. Teaching awards and honors, including those of
your students.
Effects to Date
1999-2000 Campus SOTL Program of Events
•
•
•
•
•
•
•
15 events - 930 total registrants
53 participants per event
515 individuals registers for one or more events
43 requests for videotapes
SOTL community formed and functioning
Developmental process for scholars in formation
Several SOTL-related research projects identified
The Program in Year 2
• 10 Events Scheduled
- 75 Participants per Event (Average over 4 events)
• More Efforts to Link SOTL With Other Initiatives
– Preparing Future Faculty
– The Concern with Research Ethics
– The Course Portfolio Project
Working At the Departmental
Level
• In the beginning:
– A concern for and
commitment to
excellent teaching
– A key group of faculty
members willing to
begin the conversation
about teaching in the
unit
• Next?
– Collecting data
– Sharing the results
– Designing follow-up to
the initial research:
• Involving others
• Extending the questions
An Example of DepartmentBased Research
• The problem: a group of faculty members
concerned that teaching excellence was
getting short shrift in the faculty evaluation
process
• The solution: appoint a Committee on
Teaching to make recommendations
Working Toward a Solution
Issue # 1: a lack of
confidence in student
evaluations
• Survey the faculty—
”What can/should
students tell us about
our teaching?”
• Develop and test a
new evaluation
instrument
Issue #2: a lack of good
procedures for peer
evaluations
• Gain a consensus for
how peer evaluation
might be made better
• Develop and test a
peer evaluation
procedure
Issue #1: Student Evaluation
• A departmental form was developed and
used along with the traditional university
form in all classes for 4 semesters.
• Results using the two forms were compared,
and statistical analyses were conduced to
determine reliability and validity.
A Brief Summary of Results
• Faculty members who
taught 10 required
undergraduate courses
were ranked in terms
of their overall student
evaluation scores.
• The differences among the
top 5 faculty were very
small, but the difference
between the top 5 and the
bottom 5 was sizeable.
• 86% of variance in the
rankings was predicted by
responses on one item: I
learned a lot in this course.
Issue # 2: Peer Evaluation
• The faculty consensus: traditional
classroom visitation is not very revealing of
anything useful. What we really need is
evaluation of course structure and materials
used in the class for teaching and
evaluation. We’d like to have periodic
external peer review of these materials.
Ongoing Concerns with Peer
Evaluation
• The biggest concern:
IT TAKES TOO
LONG TO PUT ALL
THIS STUFF
TOGETHER!
• What are we doing?
We’re working toward
the development of
course portfolios, to be
done over time and
turned in by faculty
members every second
or third year, not every
year.
The Campus Tie
• The departmental course portfolio effort
should eventually tie in with a campus-wide
effort now underway, conducted by an
interdisciplinary team in conjunction with a
Pew Foundation funded project directed by
Dan Bernstein at the University of
Nebraska. For more information about this
project, see http://www.unl.edu/peerrev/
TASK: Refining Your Own Campus
SOTL Program or Plan
Think for a minute about your own campus. What
administrative priorities or structures might help in
setting up a campus level SOTL program? What
hurdles might need to be overcome on your
campus?
Other Campus Models
•
•
•
•
•
•
Elon College
The Citadel
Rockhurst University
Abilene Christian University
Notre Dame University
Middlesex Community College
Elon College
• Multidisciplinary, Multiyear, $72,000
Investment
–
–
–
–
$6000 projects in each of 3 years
Projects directed by faculty-student research teams
Learning for BOTH student and teacher
Eight projects selected in years 1 and 2
• Create intellectual engagement
– New thinking in diverse fields
– Application of learning to life
– Opening spaces for reflective integration
The Citadel
• Mission
– Increased campus
awareness of and
participation in SOTL
• Focus
– Communication,
Resources and
Continuing Education
– Self-selected research
projects
• Highlights
– Biweekly, participatory meetings with
assignments
– http://www.citadel.edu/carnegie
• Effectiveness
– 15% of full-time, tenure-track faculty
at bi-weekly meetings
– 12% of full-time, tenure track faculty
in classroom research
• Administrative Support
– Attendance at functions
– Financial support
Rockhurst University
• Beginnings
• Products
– (Fall 1998) All University Symposium
– (Spring 1999) Follow-up Symposium
Year-long Carnegie faculty seminar
– (2000-2001) Carnegie faculty seminar •
continued
• The Rockhurst “Carnegie Seminar”
–
–
–
–
Central Questions
Seminar Members
Discussions
Methods
– Formal Letter on SOTL
– Faculty SOTL Projects
Selected Key Issues and Observations
– Obstacles to Discussion
– Interdisciplinary/Collaborative
Approaches
– “Where's the beef?”
– “Scholarly Teaching” as best first path
– To be a good consumer of the SOTL
Abilene Christian University
• 19 Faculty doing SOTL Projects
• Strong Institutional Support
– Stipends for materials & resources
– Travel to teaching-related conferences
– Ongoing peer meetings,videoconferences
University of Notre Dame
Initial campus conversations with 90 leaders
– SOTL needed support
– RFP resulted in 9 funded SOTL projects
Sample research question: Do new teaching methods in intro
engineering affect students’ learning?
Support for SOTL teams includes these elements:
–
–
–
–
–
$5,000 per team for student time, equipment, supplies, faculty time
Consulting with methodology experts
Group meetings 2x/semester for mutual support
Help in dissemination of results
http://www.nd.edu/~kaneb/Carnegie.html
Task: Campus Models for SOTL
List 2 features of the models you have just
seen that might be most applicable to your
campus.
Working at the Level of the
Individual Researcher
Deploying Available Resources: Some
Examples
•
•
•
•
Expand rewards
Travel grants
Presentation/research grants
Visibility for contributions
Task
"MINUTE PAPER"
1. What were the 2 most important points
presented in the first half of this workshop?
2. What 2 things would you most like to learn
tomorrow?
.
How Could I do Scholarship of
Teaching and Learning?
• Different Genres of SOTL
–
–
–
–
–
Reports on Particular Classes
Reflections on Years of Teaching
Larger Contexts—Comparisons
Formal Research
Meta-Analyses
Task
Which 3 of these genres would be easiest for
faculty on your campus to do?
Approaches to Research
Classroom vs. Traditional Research
Classroom
Traditional
Origin
Professor’s practice
State-of present research
Purpose
Obtain knowledge applicable
in limited circumstances
Build or verify theory
Requirements
Specialized training not
essential
Specialized training usually
essential
Benefits
Students and Professor
Field and Researcher
Action Research
“Research carried out by practitioners with a view
to improving their professional practice and
understanding it better.”
Quoted in Borg, Gall & Gall Applying Educational Research 3rd Ed. Longman, 1993 p. 390
Classroom Research
“Classroom Research is not traditional research conducted
in or on classrooms. It is a specific methodology designed
for discipline oriented teachers without training or
experience in the methods of educational research.
Classroom Research is ongoing and cumulative intellectual
inquiry by classroom teachers into the nature of teaching
and learning in their own classrooms. Inquiry into a
question about how students learn typically leads to new
questions and thus to continual investigations through
classroom research.”
Cross, K.P. & Steadman, M. Classroom Research, Jossey-Bass, 1996 p. xviii
IMPROVING TEACHING AND LEARNING
VIA CLASSROOM RESEARCH
IMPETUS
Goals
Questions
INFORMATION
GATHERING
Formative
Assessment
Summative
Assessment
Recorded
Observation
Existing
Scholarship
Institutional
Database
APPLICATION
Improved
Teaching and
Learning
EFFECT
Analysis
Reflection
Synthesis
SCHOLARLY
PRODUCTIVITY
Classroom Assessment
Classroom assessment is systematic and formative
• Class is the unit of measurement rather than the individual
• Conditions of learning may be assessed rather than student performance.
• Right and wrong are not the emphasis.
• Unexpected rather than expected responses are often most useful.
IMPETUS
Goals
Questions
INFORMATION
GATHERING
Formative
Assessment
Summative
Assessment
Recorded
Observation
Existing
Scholarship
Institutional
Database
EFFECT
Analysis
SCHOLARLY
PRODUCTIVITY
Reflection
Synthesis
APPLICATION
Improved
Teaching and
Learning
Angelo, T. & Cross, K.P., Classroom Assessment Techniques: A
Handbook for College Teachers (2nd Ed) Jossey-Bass (1993)
Effective Grading
Primary Trait Analysis (PTA)… building scales that make performance criteria
explicit in order to categorize/classify student work.
– Provides a documentary source for changes in student learning.
– Improves grading.
IMPETUS
Goals
Questions
INFORMATION
GATHERING
Formative
Assessment
Summative
Assessment
Recorded
Observation
Existing
Scholarship
Institutional
Database
EFFECT
Analysis
SCHOLARLY
PRODUCTIVITY
Reflection
Synthesis
APPLICATION
Improved
Teaching and
Learning
Walvoord, B. and Anderson, V., Effective Grading: A Tool for Learning and Assessment, Jossey-Bass 1998.
The Course Portfolio (I)
“I was familiar with teaching portfolios … but thinking about teaching as scholarly inquiry began
to lead me in the direction of something I had not seen anyone else doing: a portfolio that focused
on the course rather than on all of one’s teaching. Being a social scientist, I began to think of
each course … as a kind of laboratory - not a truly controlled experiment of course but as a setting
in which you start out with goals for student learning, then you adopt teaching practices that you
think will accomplish these and along the way you can watch and see if your practices are helping
to accomplish your goals, collecting evidence about effects and impact.”
IMPETUS
Goals
Questions
INFORMATION
GATHERING
Formative
Assessment
Summative
Assessment
Recorded
Observation
Existing
Scholarship
Institutional
Database
EFFECT
Analysis
SCHOLARLY
PRODUCTIVITY
Reflection
Synthesis
APPLICATION
Improved
Teaching and
Learning
W. Cerbin quoted in Hutchings, P. (Ed.) The Course Portfolio, AAHE 1998
CLASSROOM RESEARCH
IMPETUS
Goals
Questions
INFORMATION
GATHERING
Formative
Assessment
Summative
Assessment
Recorded
Observation
Existing
Scholarship
Institutional
Database
EFFECT
Analysis
SCHOLARLY
PRODUCTIVITY
Reflection
Synthesis
APPLICATION
Improved
Teaching and
Learning
Cross, K.P. and Steadman, M., Classroom Research, Jossey-Bass 1996
Task
Which kind of information gathering
(previous slide) is most common on your
campus? Which kind is most needed?
Why?
Asking the Question
Framing Research Questions
(Goal Approach)
1.
Define a goal
2.
Answer questions about the goal
3.
Create a one (or two) sentence summary of the specific goal
4.
Ask “what evidence would reveal
-the present state?”
-that the goal is achieved?”
5.
Write possible research questions
Framing Research Questions
(Issue Approach)
Criteria
• Investigable (not necessarily empirical)
• Bounded and well-defined
• Significant (not necessarily statistically)
•
•
•
•
Considerations
Length of time needed
Complexity of procedures
Availability of subjects
Availability of support (resources, personnel, funds)
Example
Less framed
Do students who help
others learn an
academic discipline
learn it better
themselves?
More framed
Do students in CMSC
250 who tutor students
in CMSC 150 perform
better on the CMSC
250 final exam than
students who do not
tutor but have similar
grades in CMSC 150?
Making Vague Questions
Answerable
• 1. Do students learn
more in small classes?
• 1. Do students in
sections of M118
enrolling fewer than
50 students perform
better on the
departmental final
exam than students
from sections
enrolling more than 75
students?
Making Vague Questions
Answerable
• What is the optimum
number of homework
assignments to give in
a beginning math
class?
• Do students enrolled in
M036 who are given a
homework assignment
every week perform
differently on the
departmental final exam
than students enrolled in
M036 who are given
homework every day?
Making Vague Questions
Answerable
• What is the effect of
the number of exams
in a course on
students’ opinions
about the course?
• Do students enrolled
in S333 who are given
6 exams per semester
evaluate the overall
course effectiveness
on BEST item #1
differently from
students in S333 who
have only 1 exam per
semester?
Task 1
1.
Write the tentative issue or question to be addressed in your
project.
2.
Form a group with two colleagues (groups of three)
3.
Discuss your tentative project with your colleagues for the
purpose of framing your goal, issue or question in the clearest
and most assessable way. Encourage your colleagues to
questions you and comment.
4.
Write your (clarified and assessable) goal/issue/question again.
5.
Assist each of your colleagues in completing steps 3 & 4.
Design Frameworks
What is a research design?
• A plan or protocol for carrying out a
research project
• An underlying scheme that governs
functioning, developing, or unfolding
Regardless of your definition, a good design
promotes efficient and successful data
gathering and analysis.
Quantitative or Qualitative
Research?
Quantitative
• Empirical, statistical
• Goal: hypothesis
testing or confirmation
• Design:
predetermined,
structured
Qualitative
• Fieldwork,
constructivist
• Goal: hypothesis
generating, making
meaning
• Design: flexible,
evolving
Comparisons, continued
Quantitative
• Sample: large,
representative
• Measures: scales,
tests, surveys
• Researcher is outsider
• Findings: precise,
reliable
Qualitative
• Sample: small,
purposeful
• Measures: interviews,
observations
• Researcher is insider
• Findings: rich, deep
Guiding Questions in Choosing
Methodology
• What approach fits your research problem?
• Do you have the skills/resources to carry out the
methods?
• Will your audience find these approaches
acceptable?
Provided by Samuel Guskin, Professor Emeritus, School of Education, Indiana University
Choosing the Measures to
Answer the Question
Examples of Quantitative Measures
•
•
•
•
•
•
course exam, project, paper scores
survey scores (Likert)
scores on standardized scales and tests
counts (participation, visits)
measures of time use
institutional research data (GPAs, grades,
admissions scores, demographics)
Examples of Qualitative Measures
•
•
•
•
•
•
observations
interviews
focus groups
student projects, essay exams (summative)
reflective statements (formative)
reports of others (counselors, etc.)
Illustrations of Qualitative and
Mixed Methods
• Qualitative case study
• Quantitative study enhanced by qualitative data
• Qualitative study enhanced by quantitative data
Provided by Samuel Guskin, Professor Emeritus, School of Education, Indiana University
Task
Design an investigation to address the
research question you framed earlier.
Summary of Standards
Clear Goals
Does the scholar state the basic purpose of
his or her work clearly? Does the scholar
define objectives that are realistic and
achievable? Does the scholar identify
important questions in the field?
Glassick, C.E., Huber, M.T., Maeroff, G.I. Scholarship Assessed Jossey-Bass (1997) pp. 22-36
Summary of Standards
Adequate Preparation
Does the scholar show an understanding of
existing scholarship in the field? Does the
scholar bring the necessary skills to his or
her work? Does the scholar bring together
the resources necessary to move the project
forward?
Glassick, C.E., Huber, M.T., Maeroff, G.I. Scholarship Assessed Jossey-Bass (1997) pp. 22-36
Summary of Standards
Appropriate Methods
Does the scholar use methods appropriate to
the goals? Does the scholar apply
effectively the methods selected? Does the
scholar modify procedures in response to
changing circumstances?
Glassick, C.E., Huber, M.T., Maeroff, G.I. Scholarship Assessed Jossey-Bass (1997) pp. 22-36
Summary of Standards
Significant Results
Does the scholar achieve the goals? Does
the scholar’s work add consequentially to
the field? Does the scholar’s work open
additional areas for further exploration?
Glassick, C.E., Huber, M.T., Maeroff, G.I. Scholarship Assessed Jossey-Bass (1997) pp. 22-36
Summary of Standards
Effective Presentation
Does the scholar use a suitable style and
effective organization to present his or her
work? Does the scholar use appropriate
forums for communicating work to its
intended audiences? Does the scholar
present his or her message with clarity and
integrity?
Glassick, C.E., Huber, M.T., Maeroff, G.I. Scholarship Assessed Jossey-Bass (1997) pp. 22-36
Summary of Standards
Reflective Critique
Does the scholar critically evaluate his or
her own work? Does the scholar bring an
appropriate breadth of evidence to his or her
critique? Does the scholar use evaluation to
improve the quality of future work?
Glassick, C.E., Huber, M.T., Maeroff, G.I. Scholarship Assessed Jossey-Bass (1997) pp. 22-36
Where to Publish and Present
How to Find Potential Sources of
External Funding
Closing Evaluation of Workshop