Emily Puckett Rodgers
Open Education Coordinator, Open.Michigan
OpenCourseWare Consortium Global Meeting 2011
Except where otherwise noted, this work is available under a Creative Commons Attribution 3.0 License.
Copyright 2011 The Regents of the University of Michigan
CC: BY-NC-SA,
Choconancy1Flickr
Measuring our Impact:
The Open.Michigan Initiative
Introduction
Evaluation team
• Emily Puckett
Rodgers
• Ted McCarthy
• Pieter Kleymeer
Evaluation: started
September 2010
http://open.umich.edu
Evaluation Design
“Through a combination of existing and new qualitative (including surveys, interviews
and user feedback) and quantitative (including web analytics, published resources)
data, we aim to measure Open.Michigan’s impact on the University
of Michigan campus and on the broader open education
landscape, as well as its progress toward overall objectives and mission.”
Mission and Objectives
Review
Environmental Scans
Document Study
and Review
Implementation
Analysis
Organizational
analyses
Research development
and deployment
Environments
Environmental Scan
General evaluations, activities in Open Education
Specific evaluation projects and documentation:
Tufts OpenCourseWare (2010)
Free to Learn Guide (2010)
MIT OpenCourseWare (2006)
Oxford’s Technology-Assisted Lifelong Learning (2010)
•
•
•
•
dScribe p
dScribes, at start of
session
Faculty
Winter
2007
Faculty who
published OER
outside of dScribe
program
Course/resources
published as OER
outside of dScribe
program
Courses Attempted
Courses Completed
by dScribe
Courses Completed
by OM Staff
Courses Completed
by Publishing
Assistant
Courses Abandoned
3
Fall 2007 Winter
2008
12
22 *
25
0
13
3
0
m
Summer Fall 2008 Winter Sum
2009***
2009
2009
2008
13
29
8
7
27
13
4
5
5
0
10
11
5
0
2
3
0
6
5
0
0
5*
1
7
3
1
0
3 1*
1
1
1
1
0
0
4
2
0
0
0 25 (started
in Summer
2009,
completed
Summer
2009Winter
2011)
25
9
0
4
0
1
0
4
0
0
0
2
•dScribe data
•Wiki
•Shared internal documents
•and spreadsheets
ses from the summer
*Does not include Medical School cour
Document Study
Strategic Vision
Open.Michigan’s Strategic Vision
Open.Michigan enables University of Michigan faculty, students, staff and
others to share their educational resources and research with the world.
Open.Michigan’s efforts contribute to two primary goals:
1.to sustain a thriving culture of sharing knowledge at U-M
2.to provide comprehensive public access to all of U-M’s scholarly output
http://tinyurl.com/openmichiganvisi
on
Strategic Plan
From the Vision
document
Measurable Activities
• Building communities of OER
producers and users
• Consulting and Outreach Services
to facilitate OER production
• Development of Processes and
Software to support OER
production and publishing
From the Evaluation
plan
Guiding Objectives
• How and Why is our OER being
used?
• Who is using our OER?
• What value does OER bring to U-M?
• In what contexts are people using our
OER?
Next Steps: Goals and Objectives document,
Communications Plan
Impact: Growth
Open.Michigan platform migrations
static Open.Michigan pages
/education OER
eduCommons OER
4/1/2008
8/27/2009
OERbit
9/16/2010
4/2011
Impact: Growth
Google Analytics
Impact: Contributions
Resource facilitation
Resources available
•Anishinaabe langauge and literature
•African Health OER Network
•“Textbook of the Future” discussions
•Center for Global Engagement
Handbook
•
•
•
•
•
~70 courses published
2 campuses
12 schools
~9 OER modules published
~48 resources deposited in Deep Blue
http://open.umich.edu/education
Impact: Contributions
Contributions to Open
Education
• OERca, released 2008 (copyright and content
clearance tool)
• OERbit, released 2011 (publishing platform)
• dScribe, process for using volunteers and
community members to clear content and
publish OER
• Code of Best Practices in Fair Use for
OpenCourseWare (2009)
• African Health OER Network
Impact: Investment
Institutional Support (U-M)
Dean support:
•Dean Woolliscroft (UMMS)
•Dean MacKie-Mason (SI)
Institutional support and
•Dean Potempa (Nursing, retired) partnerships:
•Paul Courant (MLibrary)
•Medical School
•School of Information
Financial support:
•MLibrary
•UMMS
•MERLOT
•Hewlett Foundation
•FAIMER
African Health OER Network:
•Open Society Institute
•Global Reach
•Local grants
•African Studies Center
•Ghana-Michigan Charter
•Ghana Emergency Medicine
Collaborative
“Having the big block M on a lot of high quality produced teaching modules
that are made available to the world… is mission consistent, so what’s the
business that we’re in here? We create and distribute knowledge.” -Paul
Courant, “Why Open is Important” interview
CTools survey
Impact: Surveys
40.7% instructors never heard of OCW
75.6% students never heard of OCW
http://tinyurl.com/ctools2010survey
Impact: Surveys
Open.Michigan and Open Educational Resource survey to U-M
Population Size
Completion Rate
Response Rate
39,958 students
6,149 faculty
2,268 staff
27.5%
30.4%
16.5%
71%
68%
55%
Themes:
•
•
•
•
•
Sharing habits
Awareness of Open.Michigan
and OER
Use of OER
Impact of OER
Support of OER efforts and
Open.Michigan
Impact: Surveys
Sharing
"I share learning materials with my colleagues to..." ? (faculty
responses)
Impact: Surveys
Use
“What type of OER have you used?” (student responses)
Impact: Surveys
(Some) Results
Themes from survey comments
• Establish Open.Michigan in main channels of UM communication
• Clarify copyright in OER and provide resources
• Consistently define Open.Michigan and OER
• Describe why/how to use/create OER
• Make short-term improvements to OER
• Encourage cultural shifts in learning
CC: BY Emily Puckett Rodgers
Next Steps: participating faculty
surveys
Next Steps
Evaluation
• Finish survey analysis
• Focus Groups
Strategy
• Goals and Objectives
• Communications plan
• Advisory committee
In the next three years, Open.Michigan will:
① Produce more and richer content as OER with the various campus
units, improve modularity, instructional design, and accessibility of UM OER
② Increase the visibility and discoverability of U-M resources
through a combination of marketing and metadata
③ Draw participants from more parts of campus to expand its
disciplinary coverage
④ Ensure OER production is an embedded part of the academic life
on campus
Resources
Use
http://tinyurl.com/omevaluation
• Evaluation wiki page
• Evaluation plan
• Survey questions
• Strategic planning
• Aggregate data
• Our Google Analytics data
• Aggregate survey results
Contact
Emily Puckett Rodgers
epuckett@umich.edu
open.michigan@umich.edu