1
Educational Change in
Academic Institutions:
Learning the Lessons of
Change at MIT
Daniel Hastings
September 2011
2
Agenda
• Why some educational innovation has occurred
– Importance of enablers
– Value of data
– Role of the innovator
• Why some educational innovation has not occurred
–
–
–
–
Idea of perspectives
Strategic design of curricular change
Political issues
Cultural issues
• Aftermath at MIT
3
Since 2000, MIT has developed almost
80 educational innovations
Subjects
1.00,* 6.001*, 6.002x,* 8.02(TEAL),*
8.224, HST582J, 18.03, Mission
200X*, Terrascope (1.016), 2.0012.006 & 18.02 Small Groups,* PE for
ME (2.993/2.THU), 6.002X, Robotics:
Science and Systems (6.188),* HST
100/527J
Miscellaneous
7.01 Concept Lab, Exploratory
Subjects, Interphase, Pass/No Record
First Year, Summer Institute in
Materials
Technologies
PIVoT, Cross Media Annotation
System (XMAS),* MetaMedia,*
PRS, iLabs,* Mathlets, Use of
PDAs, XMAS
Programs
The Undergraduate Exchange
with University of Cambridge,*
Residence-Based Advising*
Space
The d’Arbeloff/TEAL classroom
(26-152)
*Denotes multi-year or multi-semester assessment. Source: TLL
4
Enablers
• Money
– $25 million from
Microsoft for iCampus
– $11 million from
donors, Brit and Alex
d’Arbeloff
• Culture that supports
faculty innovation
• At least a few faculty
innovators
• Dedicated organizations
– Teaching & Learning Lab
• Provides data-driven
assessment and expertise
in STEM pedagogy
– Office of Educational
Innovation & Technology
• Develops educational
technology for faculty
– Office of Faculty Support
• Provides support for
curricular changes
Unlocking Knowledge
What is OCW?
MIT OpenCourseWare IS NOT:
• An MIT education
• Intended to represent the
interactive classroom environment
• Degree-granting
MIT OpenCourseWare IS:
• A Web-based publication of
virtually all MIT course content
• Open and available to world
• A permanent MIT activity
5
Unlocking Knowledge, Empowering Minds
Unlocking Knowledge
2,075 Courses
• 2,075 Syllabi & reading lists
• 17,531 lecture notes
• 9,460 assignments
• 980 exams
• 705 projects
Many include:
• Audio/video (~100)
• Complete texts (~30)
• Simulations/animations
http://ocw.mit.edu
6
Unlocking Knowledge, Empowering Minds
7
Why it works – faculty view
• Most popular course is 6.00 Introduction to Computer Science
• Why it works
– 1) MIT likes to do leadership things
– 2) The faculty came to believe that it was not giving away the
essence of an MIT education
– 3) The OCW infrastructure meant that there is no additional
work required of the faculty in giving a course to OCW
– 4) The faculty don’t have to answer questions from people
who read the OCW courses so there is not an ongoing issue
– 5) Faculty still can publish books which belong to them
• Most common faculty use
– Looking to see what colleagues are teaching
8
Some iCampus projects
• iLabs – students can use web browsers to design experiments
and collect data from distant laboratory equipment
• iMOAT – the web is used to manage the process of large-scale
assessment of student writing
• TEAL – two terms of introductory physics have been redesigned
around inquiry, discussion, experimentation, and visualization
• XMAS – students can ‘quote’ video legally in their online
discussions, presentations, and projects about films in courses
such as Shakespeare
• xTutor is to be a tool kit for creating online courses; its strength
is checking computer programming homework and providing
feedback
9
Collection and use of data
• Collection and use of data is necessary (but
not always sufficient) to show the efficacy of
various educational innovations
• Maps well to the culture of MIT as a scienceand engineering-oriented institution
Learning gains <g>.
0.5
0.4
0.3
0.2
0.1
0
Low
Intermediate
Lecture Students 2002
Learning gains <g> = %Correct
post-test
- % Correct
100% - %Correct pre-test
High
TEAL Students 2003
pre-test
Source: Dori, Y. & Belcher, J. (2005) “How Does Technology-Enabled
Active Learning Affect Undergraduate Students’ Understanding of
Electromagnetic Concepts?” The Journal of the Learning Sciences,
14(2):243-279.
10
TEAL increased learning gains
0.6
11
Increases seen long term
70
60
50
40
Scores
Control
Experimental
30
20
10
0
n=52 control
Pre Test
Post Test
Retention
Gains
n=120
experimental
Source: Dori, Y.J., E. Hult, L. Breslow, & J. W. Belcher (2005). “The Retention of Concepts from a Freshmen
Electromagnetism Course by MIT Upperclass Students,” paper delivered at the NARST annual conference.
Learning Analytics – Assessing
Learning
• Learner Analytics
– e.g. Gates Next Generation Learning Challenge
– Predict/Reduce drop, fail, withdraw rates in community
colleges, especially in gateway courses
• Instead Assess the Learning Process
– Detailed student knowledge (then tutor weaknesses)
– Reveal Student Habits
– Measure Effectiveness of Instructional Elements
12
Eductional Data Mining: Tutors give DATA!
Tutor > 105 Bytes vs. Gradebook < 102
Fine Grain Assessment – Holy Grail
• Assessment of Detailed Skills of Student
• Formative Assessment for the Teacher
• Ultimately will guide individual tutoring
Habits of Mind and Behavior
• What Habits help/hinder learning??
• e.g. Homework copying reduces learning
Improve/Find Highly Instructional Activities
• Analysis of Student Use Data Improves Activity
• Which Course Activity Correlates with Learning?
• What type of activities give what types of learning?
13
Detailed Skill Profiles
Measured skill of each student in various topics – 100 is average
Red is below average, Green is above average
This is experimental skill-book in cybertutor.mit.edu (D. Pritchard’s online tutor)
14
Improving Efficacy of Assessment Items
15
Revising the questions based on analysis of the student use data allows dramatic improvement
in the performance of students. From 2001 to 2002 the percentage of students who
requested the solution went down by ~ 50% (from 12% to ~6%). The number of wrong
answers per question decreased by 40%
16
Innovations for
which improved
learning can be
demonstrated did
one or more of
the following . . .
17
Employed pedagogies of
engagement
18
Confidence in abilities increased in 6.01
with introduction of active learning
Programmin
g
3.16
Ability to
learn new
programming
language on
own
2.67
Source: O’Leary, L. “6.01 Class Experience Survey Synopsis, Spring 2008,”
6/16/08.
n=
153
19
Motivated students by
grounding learning in realworld problems or projects
Students’ confidence in crucial abilities
strengthened in PBL freshmen subjects
Skill
% who said skill strengthened
by PBL
Break a problem into key components
87%
Identify tasks, resources, time, and people
to solve a problem
79%
Select and combine the best ideas into a
better approach
93%
Recognize constraints that limit a solution
91%
Greater confidence in ability to handle a
difficult and ambiguous task*
84%
Able to use their own initiative*
87%
See connections between the PB class and
what they learned in other subjects*
81%
Source: Office of Faculty Support, “Project-Based Pilot
Assessment Results,” 10/10/07.
* In comparison to regular
subjects
n=90
20
21
Provided multiple
opportunities for
‘practice at retrieval’
22
Performance improved in 16.100
Pedagogical changes included: concept questions and mini
lectures in class; look ahead (graded) homework; written takehome exams; oral exams; semester-long, team-based project
Source: Darmafol, D. “Impact of Pedagogical Changes in an Undergraduate
Aerodynamics Subject: A Summary,” 2004.
23
Educational technology as an
enabler
• The merger of research and education tools
– Faculty develop/want to develop research tools
– Some of these tools can be used (appropriately
modified) in education
• Simulations and animations
• Remote laboratories
24
Research tools for learning
• Early exposure to research
tools and experience
This type of hands-on
interaction with the
molecule provides levels of
insights that are not
possible by viewing static
images on a page on a
computer screen
~ Prof. Graham Walker
• Used by 1000 MIT students
students; 300 high school
students
•
StarBiogene, StarHydro,
StarHPC
StarBiochem
24
25
The role of the innovator: The example
of iLabs created by Jesus del Alamo
• Real laboratory accessed through Internet
• From anywhere at any time
• Not a “virtual laboratory” (simulations)
26
iLabs at MIT
Dynamic signal
analyzer (EECS,
ELVIS (EECS, deployed
2006)
deployed 2004)
Heat exchanger
(Chem. Eng.,
deployed 2001)
Microelectronics
device
characterization
(EECS, deployed 1998)
Polymer
Crystallization
(Chem. E., deployed
2003)
Spectrometer
(Nuclear Eng.,
deployed 2008)
Shake Table (Civil
Eng., deployed 2004)
Force on a Dipole
(Physics, deployed
2008)
27
Educational change that has not
occurred at MIT
• Idea of perspectives
• Strategic design of curricula change
• Political issues
• Cultural issues
Perspectives
(John Van Mannen@ Sloan School)
Perspectives are organized ideas (e.g., metaphors) that
fundamentally shape our understanding of things and events.
• They determine what data you see (hear, feel) in the
organization
– What questions you ask
– Where your attention lies
• They determine how we interpret the data we see
– They prioritize the data we receive and shape our
actions
• No single perspective is adequate
– It is easy to get locked into a single perspective,
but difficult then to deal with complexity
28
29
Three perspectives
Strategic Design
Political
Organizations are machines
Organizations are contests
An organization is a mechanical system
crafted to achieve a defined goal. Parts
must fit well together and match the
demands of the environment.
An organization is a social system
encompassing diverse, and sometimes
contradictory, interests and goals.
Competition for resources is expected.
Action comes through
planning.
Action comes through power.
Cultural
Organizations are institutions
An organization is a symbolic system of
meanings, artifacts, values, and routines.
Informal norms and traditions exert a
strong influence on behavior.
Action comes through habit.
30
Let’s get specific:
Lessons learned
from the past
decade
at MIT.
31
MIT’s Educational Mission
In 1998 the Task Force on Student Life and Learning, as part of
two years of intensive study, consultation, and reflection,
developed a formal statement of MIT's educational mission:
The Massachusetts Institute of Technology is devoted to the
advancement of knowledge and education of students in areas that
contribute to or prosper in an environment of science and
technology. Its mission is to contribute to society through excellence
in education, research, and public service, drawing on core strengths
in science, engineering, architecture, humanities and social sciences,
and management. This mission is accomplished by an educational
program combining rigorous academic study and the excitement of
research with the support and intellectual stimulation of a diverse
campus community.
The message matters...
32
Senior leadership matters
• The presidentially appointed Task Force on Student Life &
Learning was charged in 1996 with undertaking a
comprehensive review of MIT's educational mission on the
threshold of the 21st century.
– President Vest asked: “What does MIT have to do to be the preeminent university in Science and Technology in 2020?”
• The final report published in 1998 provided insight into the
principles that define MIT and the attributes of an
educated individual.
• In addition, the Task Force made recommendations
regarding the General Institute Requirements (GIRs),
advising, the first year, teaching, and undergraduate
research.
The core curriculum remained
untouched
33
• The first Task Force focused on student life
issues that had become more overtly challenging.
• Advising was perceived as in serious need of
improvement.
• The core curriculum was not seen as ‘broken.’
• And after two years of hard work, that group
needed reinforcements.
And so . . .
34
MIT Task Force on the Undergraduate
Educational Commons
The Massachusetts Institute of
Technology established a
Presidential Task Force on the
Undergraduate Educational
Commons in order to undertake
a fundamental, comprehensive
review of the common
educational experience of our
undergraduates in the early
years of the twenty-first century.
35
Task Force overview
• The Task Force on the Undergraduate Educational Commons
was a committee of two dozen MIT faculty members and
undergraduates.
• The Task Force comprehensively reviewed MIT’s ‘General
Institute Requirements’ (GIRs), the rigorous foundation in natural
science, mathematics, technology, humanities, arts, and social
sciences that forms the core curriculum of every undergraduate’s
MIT education.
• The Task Force Report affirmed the many ways in which this
common curriculum has successfully prepared MIT’s graduates
for a lifetime of learning and leadership, but it also recognized
that changes in the wider cultural context required curricular
change.
36
Strategic design
• After 2.5 years of work, a committee of 24
faculty presented a new core curriculum to the
entire MIT faculty
• Tried to get input via
– Departmental meetings
– Town Hall meetings
– Presentations to senior administrators and School
Councils
– E-mail to faculty/students and a website
– An independent student committee
37
Curriculum design changes
• The portion of the General Institute Requirements that focuses
on science and technology should provide greater flexibility in
the choice of classes in the fundamental sciences while retaining
the rigor that has been the historic hallmark of these classes.
• The Humanities, Arts, and Social Sciences Requirement should
be clarified in order to provide a rigorous foundation in the study
of human culture, expression, and social organization.
• MIT should make it clear that acquiring experience living and
working abroad is an essential feature of an undergraduate
education, and work to expand current international education
programs that have proven successful in the MIT environment,
and develop strategies to create other [global] opportunities.
[http://web.mit.edu/committees/edcommons/documents/task_force
_report.html]
38
Some recommendations were
implemented: Online subject evaluations
• MIT did not have one uniform subject evaluation system. Some
departments ran their own online subject evaluations, while most
used paper forms provided and processed through the Office of
Faculty Support.
• The ‘system’ was a patchwork of processes that did not provide
the tools needed by faculty and academic administrators to
compare or discern patterns in student responses over the
semesters.
• Gradual scale-up & careful piloting has led to positive community
response.
• Expected benefits of the online system include:
– More thorough data collection and longitudinal analysis
– Simplified administration
– Better reporting capabilities
39
New GIRs: Full Model
HUMANITIES, ARTS AND SOCIAL SCIENCES REQUIREMENT (8 subjects)
Required Subjects
Freshman Experience
Expository Writing (if needed)
Foundational electives in HASS (1 subject from 2 out of 3 categories)
Humanities
Social Sciences
Advanced Subjects (3-4 subjects)
Concentration
HASS Electives
Arts
SCIENCE/MATHEMATICS/ENGINEERING REQUIREMENT (8 subjects)
Required Subjects
Mechanics
Single-Variable Calculus
Multi-Variable Calculus
Distribution Subjects:
1 from each of 5 categories of the 6 below
Mathematics
Differential
equations,
linear algebra,
probability and
statistics…..
Physical Sciences
Electricity and
magnetism,
Physics of…
Chemical Sciences
Solid-state
chemistry,
Chemistry of…
Life Sciences
Molecular Biology,
Biology of…
Computation &
Engineering
Algorithmic Thinking,
Engineering Practice
Freshman
Experience
Project-Based
Subjects in
Engineering, Science
and/or Design
40
Political perspective: School of Science
• New Science Math and Engineering (SME) core called for more
‘flavors’ for science, and for engineering to have a project-based
experience, as well as a design class in the core
• Each 12-unit SME GIR element should be offered in a variety of
flavors that would share core content ( 6 units).
• Disadvantaged in this would be the science departments
particularly physics, math, and chemistry
– Budgets are influenced by how many undergraduates
departments teach; maybe even future faculty slots
– Math and chemistry support many graduate students as TAs
• These departments all opposed the changes
• Dean of Science was in favor but could not carry the School
41
Political perspective: School of Engineering
• Engineering did not care enough to push hard
• Engineering could not or would not articulate
a coherent core subject with broad support
• There was disagreement on the intellectual
content of the project-based course
• Dean of Engineering was unable to get
Engineering faculty buy-in
42
Political perspective: School of Humanities
• Humanities, Arts and Social Science requirement
would be taught entirely in that School, so while
there were some who were disadvantaged they
could be accommodated
• Proposal was seen as providing more coherence,
which was endorsed by many
• Advocates worked to create trust in advance, to
speak within the community and build
consensus, both bottom up and top down
• Dean of Humanities, Arts and Social Science
was visibly behind the proposal
43
Cultural perspective
• The proposed SME changes raised deep
questions
– What is modern science (and how can you devalue my
science by throwing it out!)?
– Who can teach this modern science?
• Can only physicists teach physics? (dept said yes)
• Can only mathematicians teach math? (dept said yes)
– Is engineering really a fundamental subject like
physics and math?
• Is there core content that is different from science? (some in
SoE said yes, around design; many in SoS said no)
Cultural perspective
• Analytic rigor is a very deep value in MIT culture
– The project-based course was heavily critiqued as
lacking rigor and unworthy of being in the core
– An engineering GIR (in design, for example) would
require some science subject to be eliminated, which
raised concerns about “watering down” the rigor of the
undergraduate curriculum
• Some engineers proposed removing
humanities or arts subjects but that School
objected (power issue)
44
Aftermath
45
• Humanities changes passed faculty vote
• Science Math and Engineering proposal did not pass
(required 60% vote)
• Engineering still not part of core, but math has
reached out to work more with engineering
• No energy for another try at MIT. Trying new ideas in
Singapore with SUTD and will bring back to MIT.
• Several departments have created new degrees and
made changes in major programs
• Emphasis on globalization is proceeding (but did not
require a faculty vote!)
46
Much curricula change has occurred
• Often at local level (individual faculty) or departments
• Funds must exist to make this happen
• Having dedicated organizations for educational
technology, pedagogy, and assessment is very
helpful
• Dissemination, scalability, and broader coordination
remain challenges
• Wholesale change requires a three lens perspective
• Change can often occur more easily outside the
mainstream
• MIT continues to be a dynamic exciting place to teach
and learn