U21 Educational Innovation Conference
University College Dublin
31 October – 01 November 2013
Exploring the Cost & Benefits of
Online Innovation
Diana Laurillard
London Knowledge Lab
Institute of Education
Exploring the costs and benefits of online
innovation in education
Outline of the argument
• Global demand for education
• Why we need to understand costs and benefits
• What it takes to teach with technology
• Teaching on the large scale
• Tools for teachers as designers
• Modelling costs and benefits
The global demand for education
By 2025, the global demand for higher education will double to
~200m per year, mostly from emerging economies (NAFSA 2010)
The new UNESCO goals for education:
• Every child completes a full 9 years of
free basic education …
• Post-basic education expanded to
meet needs for knowledge and skills …
(Draft for UNESCO post 2015 goals)
 Implying significant graduate and teacher training
growth for this level of schooling, FE and HE
1:25 staff:students??
Why we need to understand costs and
benefits
The global demand for HE is a demand for new pedagogies
capable of educating millions – far from the current model
Teaching has to adapt continually to rapid changes in
opportunities from digital technology innovation, changing
student needs, capabilities and expectations
 teachers have to discover the digital pedagogic forms
that scale up for large-scale high quality teaching
The iterative and adaptive nature of design must be at the core
of teaching innovation in HE in this rapidly changing world
Understanding high quality TEL
Preparation time (fixed costs)
MOOC vs standard online course
• Adaptive feedback (sim/models/games)
• Expositions (lecture videos)
• Automated grading (MCQs, quizzes)
• Readings (pdfs)
• Collaboration activities (wiki)
• Peer group discussion (forums)
• Peer grading against criteria (tests)
• Tutored discussion (forums)
• Tutor feedback (e-portfolio)
Support time (variable costs)
What it takes to teach with technology
The teaching workload is increasing in terms of
Planning for how students will learn in the mix of the physical,
digital and social learning spaces designed for them
Curating and adapting existing content resources
Designing the activities, tools and resources that afford all
types of active learning
Personalised and adaptive teaching that improves on
traditional methods
Providing flexibility in blended learning options
Guiding and nurturing large cohorts of students
Using learning technologies to improve scale AND outcomes
BUT:
Institutions and teachers do not typically plan for the teaching
workload implied by these learning benefits
nor for the need to collaborate to innovate with technology
The MOOC as ‘large-scale’ pedagogy
MOOCs are not large scale – Duke University
Completed = 2% of enrolment, 25% of ‘engaged’
Duke University Report 2012
The MOOC as ‘large-scale’ pedagogy
Average student numbers per course - Edinburgh
Enrolled
51500
Accessed Week 1
20500
Engaged Week 1
15000
Week 5 asst's
6000
Statement of Accomplishment
5500
0
10000 20000 30000 40000 50000 60000
Completed = 10% of enrolment, 37% of ‘engaged’
MOOCs @ Edinburgh 2013 – Report #1
The MOOC as undergraduate education
Not for undergraduates
72%
have
degrees
Enrolled students
Duke University Report 2012
The MOOC as undergraduate education
Not for undergraduates
PG degree
40%
Degree
30%
College
Enrolled students
70%
have
degrees
17%
School
10%
Less than high school
3%
0%
10%
20%
30%
40%
50%
MOOCs @ Edinburgh 2013 – Report #1
What it takes to teach
a basic MOOC vs the Duke MOOC
Total teaching time
3000
2500
2000
Duke MOOC
1500
Basic MOOC
1000
Prep
time =
420
500
The variable cost of
high quality teaching
does not achieve
economies of scale
if you maintain the
same pedagogy
0
50
500
5000
Preparation time = 420 hrs
Teaching time
50
500
5000
Duke MOOC
20 hrs
200 hrs
2000 hrs
Basic MOOC
0.00
0.00
0.00
Duke University Report 2012
Modelling the benefits and costs
It’s important to understand the link between the
pedagogical benefits and teaching time costs of
online learning – especially for the large-scale
What are the new digital pedagogies that will
address the 1:25 student guidance conundrum? –
How to shift variable cost support to fixed cost
support?
Can we develop a viable business model that will
make HE affordable for novice learners?
Pedagogies for supporting large classes
Concealed MCQs
The (virtual) Keller Plan
The vicarious master class
Pyramid discussion groups
Conceal answers to question
Ask for user-constructed input
Introduce
content
Reveal multiple
answers
Self-paced
Ask user to practice
select nearest fit
Tutor-marked
test
240
individual
students produce
Tutorial
for
5
representative
Student
becomes
tutor for credit
response
to open question
students
Until
half
class and
is tutoring
thejoint
rest
Pairs
compare
produce
Questions
and guidance
represent
response
all students’ needs
Groups of 4 compare and produce
joint response and post as one of
10 responses...
6 groups of 40 students vote on
best response
Teacher receives 6 responses to
comment on
Pedagogies for supporting large classes
Concealed MCQs
Laurillard, 2002
The (virtual) Keller Plan
Keller, 1974
The vicarious master class
Mayes et al, 2001
Pyramid discussion groups
Gibbs et al, 1992
The traditional pedagogies for large classes could be
redesigned as digital formats
Tools for academics as learning designers
1. Library of learning designs indexed by learning outcome & topic
2. Three different subject instances for each to promote migration of
good pedagogy across domains
3. Academic ‘adopts’ a design and adapts it as needed using edit
tools & links to other resources – creating a computationally
interpretable design
4. Library of OERs for academics to link to from learning designs
5. Feedback on ‘learning experience’ created, total learning time, and
teacher preparation and contact time
6. Prompt to include ‘production’ activity for collecting learning
analytics on outcomes
7. Design sent to Moodle to test with students – collects data
8. Students can annotate design for detailed evaluation
9. Academic redesigns as needed – tests again – publishes to Library
Tools for teachers as learning designers
Teachers as design scientists need the tools for innovation
To find or
create new
ideas
Adopt
Adapt
Test
http://tinyurl.com/ppcollector
To collect
learning
analytics
Redesign
Analyse
Publish
Creating knowledge about effective
blended and online pedagogies
Tools for academics as learning designers
http://tinyurl.com/ppcollector
Tools for academics as learning designers
1.
2.
3.
4.
5.
6.
7.
8.
9.
Library of learning designs indexed by learning outcome & topic
Three different subject instances for each design to promote migration
of good pedagogy across domains
Academic adopts’ a design and adapts it as needed using edit tools &
links to other resources – creating a computationally interpretable
design
Library of OERs for academics to link to from learning designs
Feedback on ‘learning experience’ created, total learning time, and
teacher preparation and contact time
Prompt to include ‘production’ activity for collecting learning analytics
on outcome
Design sent to Moodle to test with students – collects data
Students can annotate design for detailed evaluation
Academic redesigns as needed – tests again – publishes to Library
The design cycle for teaching
Build on others’
tested designs
Select
Adopt
Redesign
existing
content
resources?
Publish
Adapt
Test
Test
Redesign
Building teaching community knowledge
Make links to
existing content
resources
Similar to the design cycle for science
What is the
teaching design
equivalent of
the journal
paper?
Select
Adopt
Publish
Adapt
Test
Test
Redesign
Building scientific knowledge
A learning design for Ed students
Make links to
existing content
resources
Add link to an
OER, e.g. a digital
tool for practice
Export to Word
or LMS/VLE
Check the feedback
on the overall
distribution of
learning activity
The learning design as a ‘content shell’ ready to receive content products?
Export to Moodle for Ed students
• Interprets metadata to assign
activity types in Moodle (or other
LMS)
• Attaches resource links
• Inserts study guidance from text
in the pattern
• Collects data on student
performance on TEL-based
activities
Reversioned for Med students
• Same pedagogical pattern
• Same study guidance except
for subject content terms and
resources
• Different resources attached
• Same type of evidence data (?)
Modelling the pedagogic benefits
A computational representation can analyse how much of
each learning activity has been designed in
Categorised
learning activities
Conventional
Acquisition
Inquiry
Discussion
Practice
Production
Blended
Acquisition
Inquiry
Discussion
Practice
Production
Analysis shows more
active learning
The Course Resource Appraisal Model …
Run No. of students
Run 1
15
No. of students
Run 2 Run 20
15
Run 3 Run 1 20
Run 2
30
Run 3
60
Run 1 Run 2 Run 3
Students 15
20
20
Run£4k
1 Run
Profit
-£27k
£11k2 Run 3
Students 15
30
60
Profit
-£27k £11k £38k
Modelling the teaching time costs
An interactive learning design tool can analyse
how much design and teaching time is needed
Design hrs
Teaching hrs
Design hrs
Teaching hrs
Yr1
Yr2
Yr3
Yr1
Yr2
Yr3
40
40
40
40
80
160
Students
Students
Managing the fixed costs of teaching
What academics need
Do publishers help?
• Create and test professional content
✓✓✓
• Curate existing professional content
✓✓✗
• Adapt and customise existing content
✓✗
• Select and organise community content
✗
• Create reusable learning designs as ‘content shells’ ✗
• Adapt and customise existing learning designs
✗
Could the learning design interface to the VLE, for creating ‘content shells’
suggest a new kind of content product?
Further details…
tinyurl.com/ppcollector
http://buildingcommunityknowledge.wordpress.com
Teaching as a Design
Science: Building
pedagogical patterns for
learning and technology
(Routledge, 2012)
d.laurillard@ioe.ac.uk
Teaching as a design science: Summary
The global demand for education requires investment in
pedagogic innovation for high quality large scale teaching
We need to design, test, use, and reuse high quality open
educational resources that amortise high fixed costs over large
student cohorts
We need to invent variable cost pedagogic innovations that
supports students at a better than 1:25 staff-student ratio
Teachers need the tools to design, test, gather the evidence of
what works, and model benefits and costs
Teachers are the engine of innovation – discovering the means
by which we fulfill our social responsibility of doing more for
less to meet that global demand
Break-out questions
1.
2.
3.
4.
5.
6.
Should our universities play any role, or take any responsibility for
meeting the global demand for HE? [Slides 2, 3]
Can universities and teachers plan for the teaching workload implied by
the learning benefits that technology can confer? [Slide 4]
Can academic teachers play a part in discovering the digital pedagogic
methods that will scale up to provide large-scale high quality teaching in
order to reduce the costs to students/government of meeting the
national demand for HE and lifelong learning? If not – who? [Slide 5, 6]
Can we use pedagogy-driven learning analytics to understand better the
relationships between teaching and learning? [Slide 6]
Can university teachers collaborate to innovate with technology? [Slide 7]
Do we know the real costs of current teaching as a business model with a
related return? How can we understand the new cost models for moving
to online courses if we do not have a clear activity-based cost model for
current teaching? [Slides 8, 9]