08 July 2013
Modelling Teaching Costs
Patricia Charlton
Diana Laurillard
London Knowledge Lab
Institute of Education, London
Overview
Overview
•
•
•
•
•
Brief introduction
Case study and discussion
Look at MOOCs and scaling up
CRAM tool
Community Knowledge and sharing learning
designs*
• Big data and Learning Analytics
• Conclusion
The
The age
age of
of the
the smart
smart machine
machine
“vision came together for me that morning I realized that the
people I had been interviewing were on the edge of a
historical transformation of immense proportions, as
important as that which had been experienced by the
eighteenth and nineteenth century workers. I saw that a
world of sensibilities and expectations was being irretrievably
displaced by a new world, one I did not yet understand”(pg
xiii)) (Zuboff,(1984) )
The global demand for education
By 2025, the global demand for higher education will double to ~200m per
year, mostly from emerging economies (NAFSA 2010)
The new UNESCO goals for education:
• Every child completes a full 9 years of free
basic education …
• Post-basic education expanded to meet needs
for knowledge and skills … (Draft for UNESCO
post 2015 goals)
 Implying significant growth in graduate numbers
to supply this level of education
“An innovation must be
But staff:student ratios in the current HE model are aligned with corporate
~1:25, which cannot meet this level of demand
interests for it to work”
(Richard Maccabee)
 Can technology help?
Supporting high quality online learning
Preparation time (fixed costs)
MOOC vs standard online course
• Adaptive feedback (sim/models/games)
• Expositions (lecture videos)
• Automated grading (MCQs, quizzes)
• Readings (pdfs)
• Collaboration activities (wiki)
• Peer group discussion (forums)
• Peer grading against criteria (tests)
• Tutored discussion (forums)
• Tutor feedback (e-portfolio)
Support time (variable costs)
The MOOC as ‘large-scale’ pedagogy
MOOCs are not large scale – Duke University
Completed = 2% of enrolment, 25% of ‘engaged’
Duke University Report 2012
The MOOC as ‘large-scale’ pedagogy
Average student numbers per course - Edinburgh
Enrolled
51500
Accessed Week 1
20500
Engaged Week 1
15000
Week 5 asst's
6000
Statement of Accomplishment
5500
0
10000 20000 30000 40000 50000 60000
Completed = 10% of enrolment, 37% of ‘engaged’
MOOCs @ Edinburgh 2013 – Report #1
The MOOC as undergraduate education
Not for undergraduates
72%
have
degrees
Enrolled students
Duke University Report 2012
The MOOC as undergraduate education
Not for undergraduates
PG degree
40%
Degree
30%
College
Enrolled students
70%
have
degrees
17%
School
10%
Less than high school
3%
0%
10%
20%
30%
40%
50%
MOOCs @ Edinburgh 2013 – Report #1
What does it take to teach a MOOC?
8 weeks, providing 50 hours learning time, including support:
Videos and pdfs
Quizzes
Wiki
Peer discussions
Peer grading
Tutored discussions
Summative assessment
High on prep time
Zero tutor contact for 42 hours
Low on prep time
High contact for 8 hours learning
420 hours to develop materials and course design
200 hours to support 8 hours for ~500 students = 1:20 staff student ratio
How does that scale up to large student numbers?
Duke University Report 2012
What it takes to teach
a basic MOOC vs the Duke MOOC
Total teaching time
“What students
need is support and
encouragement”
(Lindsay Jordan)
3000
2500
2000
Duke MOOC
1500
Basic MOOC
1000
Prep
time =
420
500
0
50
500
5000
Preparation time = 420 hrs
Support time
50
500
5000
Duke MOOC
20 hrs
200 hrs
2000 hrs
Basic MOOC
0.00
0.00
0.00
The variable cost of
high quality support
does not achieve
economies of scale
if you maintain the
same pedagogy
Modelling the benefits and costs
• It’s important to understand the link between the
pedagogical benefits and teaching time costs of
online learning – especially for the large-scale
• What are the new digital pedagogies that will
address the 1:25 student support conundrum?
CRAM
Tool
CRAM TOOL
• http://modellingteachingcosts.wordpress.com
/cram-tool/
Pedagogies for supporting large classes
Concealed MCQs
The (virtual) Keller Plan
The vicarious master class
Pyramid discussion groups
Conceal answers to question
Ask for user-constructed input
Introduce
content
Reveal multiple
answers
Self-paced
Ask user to practice
select nearest fit
Tutor-marked
test
240
individual
students produce
Tutorial
for
5
representative
Student
becomes
tutor for credit
response
to open question
students
Until
half
class and
is tutoring
thejoint
rest
Pairs
compare
produce
Questions
and guidance
represent
response
all students’ needs
Groups of 4 compare and produce
joint response and post as one of
10 responses...
6 groups of 40 students vote on
best response
Teacher receives 6 responses to
comment on
Pedagogies for supporting large classes
Concealed MCQs
Laurillard, 2002
The (virtual) Keller Plan
Keller, 1974
The vicarious master class
Mayes et al, 2001
Pyramid discussion groups
Gibbs et al, 1992
The traditional pedagogies for large classes could be
redesigned as digital formats
Tools for teachers to share ideas
Teachers as designers need the tools for innovation
To find or
create new
ideas
Adopt
Adapt
Test
http://tinyurl.com/ppcollector
To collect
learning
analytics
Redesign
Analyse
Publish
Creating knowledge about effective
blended and online pedagogies
Tools for teachers to share ideas
http://tinyurl.com/ppcollector
Academics sharing their best designs
A library of
patterns to
inspect
Defining the metadata of their pedagogies
Assigned metadata on
• learning type
• group size
• duration in minutes
• teacher contact/not
• resources attached
• evidence of learning
Adopt and adapt design for Ed students
Share the
pattern
Export to
Word
[Moodle]
Specify the
duration of the
activity in minutes
Adjust the type of
learning activity.
Edit the instructions.
Check the feedback
on the overall
distribution of
learning activity
Adopt and adapt design for Ed students
“… I can pick up
bits of what you
do” (Kevin Ashley)
Export to Moodle for Ed students
• Interprets metadata to assign
activity types in Moodle (or other
LMS)
• Attaches resource links
• Inserts study guidance from text
in the pattern
• Collects data on student
performance on TEL-based
activities
Reuse for Med students in PPC
Explain how to optimise the inputs to a learning design tool to
achieve a well-balanced learning design
Explain how to optimise the inputs to a patient simulator to
achieve the ideal blood pressure
With your partner select different inputs to the learning
design tool – can you improve on your previous results?
With your partner select different inputs to the patient
simulator – can you improve on your previous results?
Reversioned for Med students
“We do not share
as much as we
should” (Nicola
Millard)
• Same pedagogical pattern
• Same study guidance except
for subject content terms
• Different resources attached
• Same type of evidence data (?)
Modelling the pedagogic benefits
A computational representation can analyse how much of
each learning activity has been designed in
Categorised
learning activities
Conventional
Acquisition
Inquiry
Discussion
Practice
Production
Blended
Acquisition
Inquiry
Discussion
Practice
Production
Analysis shows more
active learning
Modelling the teaching time costs
Modelling an IOE course over 3 years:
the Course Resource Appraisal Model
Prep hrs
Prep hrs
Support hrs
Support hrs
Yr1
Yr2
Yr3
Figure 2(a) Teaching time for a
course with 40 students each year,
gives profits of
-£12000
£5000
£8000
Yr1
Yr2
Yr3
Figure 2(b) Teaching time for a
course with 40, 80, 160 students,
gives profits of
-£12000
£13000
£35000
+ need to model differences in administrative support costs for step changes
Big data and digital knowledge and
Learning Analytics
‘Not everything that can be counted counts, and not
everything that counts can be counted.’ Attributed to
Albert Einstein
• In a BBC Radio 4 interview, Kenneth Cukier , when discussing Big
Data stated “Let the data speak for itself”.
• “more data does not mean more knowledge” (Dr. Tiffany Jenkins ).
• Cukier describes an example from Google predicting the flu
outbreak in the US. He explains that a correlation experiment was
run using millions of terms where they used lots of mathematical
models:
ʻ...what they were able to find was they were able to correlate in a
very short period of time where flu outbreaks were going to be in
quasi real timeʼ.
Modelling the potential pedagogical benefits
Categorised
learning activities
Conventional
Acquisition
Inquiry
Discussion
Practice
Production
Analysis shows more
active learning
To practice-based production
From Big Data to Intelligent Context
From Big Data to Intelligent Context
Connecting
Schools & Institutes
Me & Myself
Family & Friends
Community
Scenario
Scenario
Scenario
I would like to learn
‘how to tell the time?’
Share a new idea
about saving energy
Generalised pattern
learning in maths
Scenario
Share my LD about
generalized pattern
learning
Ubiquitous applications
Pervasive apps
Knowledge product tool
Learning Designer
Tell the time app
Post my new idea and
share with my friends
MiGen -eXpresser
Share my design with
teachers and learners
Scenario
I would like to be more
ECO friendly
Scenario
I want to help my child
improve his social
skills
Scenario
Scenario
Social interaction
learning for children
with Autism
Let parents, teachers
and children know
about new tools
Social & Cultural apps
Knowledge apps
ECHOES II
Learning Designer
Take the energy habit
challenge & see how
you can improve
ECHOES can help
Use ECHOES tool to
support learning
Share my designs that
use the new tools
Community Knowledge
Context-aware Engine
Engine
Rules
Domain Concepts
My Concepts
Community Shared concepts
My Community
Frequently connect with
and share
Personal
Organisations
Community network
Intelligent Context
Analysis
to Learning
enable
Personal, Teaching
Knowledge
Sharing and
Experience
and Learning Analytics
Jack sees/checks his ECO performance
using TAKTEEN
1
(Personalized & social)
Jack gets some new music resource through
Trend analyzer (RSS)
3
(Sharing & building community knowledge)
4
2
Jack shares his current
performance .
He is working with his friends and
family to see how to improve their
impact
5
6
Jake updates his teaching for next week
Includes the new music resource & sends
out an update to students
(community knowledge builder sharing)
Jake discovers a
new resource .
to improve social
interaction children with
Autism Spectrum
Disorder.
Shares with family &
friends
(Sharing & building
community
knowledge)
Jake sees students feedback
Students are having difficulty
in explaining abstraction.
Jake searches for resource
(Knowledge building re-use)
(social community aware)
Jake finds a suitable maths tool
Jake sends knowledge product
to the students eXpresser to test out.
(Knowledge building re-use)
8
7
9
Jake listens to examples of new
music resource.
(Family domain based sharing )
At Home
Jake sees feedback about maths resource
Students have now started
A discussion group using the tool in second life
(Ubiquitous knowledge on any device)
On the move
At Work
The uncomfortable truths of education
economics
(No university or college finance director addresses these!)
Scaling up to large numbers will never improve
theleaders
per-student
“Senior
should
support costs…
be modelling what they
expect from
theirthat
staff”
…unless we come up with some clever learning
designs
(Cathy Walsh)
support at better than the 1:25 ratio
 We need to invest in teacher innovation to make the best use of
our teaching resource for students’ outcomes
 Teachers sharing innovations will improve knowledge, quality
and value for students’ money
SomeLessons
Lessons Learnt
Some
Learnt
• Get to know your user group or intended user group and have them
involved from the beginning;
• Technology evolves at a speed: - is your problem and solution tied to a
particular environment;
• Understanding the advantages and disadvantages of the approach
being used;
• What kind of data or models exist about the problem space?
• What solutions have been tried in the past and why do they not work?
Know the landscape
• What to do when you know you are on the right path but few agree?
• Solutions rarely, if ever, sit in isolation
Food
for
Thought
Food for thought
• Your best resource and solutions are the people you work
with! Technology is no substitute
• Context is everything but you need the right context;
• Building systems that contain “intelligence” is challenging
so resilience and robustness are a must;
• Be prepared to throw a solution away and start again.
Failure is a prerequisite to success!
• Know what you are contributing and why!
Further details…
http://buildingcommunityknowledge.wordpress.com/
http://tinyurl.co,/ppcollector
“Science moves
faster with open
access” (Alicia
Wise)
Teaching as a Design Science:
Building pedagogical patterns for
learning and technology
(Routledge, 2012)
d.laurillard@ioe.ac.uk
The pedagogies for large-scale student
guidance
The global demand for education requires investment in
large-scale pedagogic innovation for MOOCs to deliver
Digital
Digital pedagogic
pedagogic innovation
innovation must
must support
support students
students at
at aa
better
better than
than 1:25
1:25 staff-student
staff-student ratio
ratio
Teachers need the tools to design, test and share the
Teachers need the tools to design, test and share the
evidence of what works, and model benefits and costs
evidence of what works, and model benefits and costs
Teachers are the engine of innovation – designing,
Teachers
the engine
of innovation
designing,
testing,are
sharing
their best
pedagogic– ideas
testing, sharing their best pedagogic ideas
 And they need your help to do it!
References
References
•
•
Charlton P., Magoulas G., Laurillard, D. (2012) Enabling Creative Learning Design
through Semantic Web Technologies, Journal of Technology, Pedagogy and
Education, 21, 231-253
Laurillard, D., (2012) “Teaching as a Design Science”, Building Pedagogical Patterns for
Learning and Technology, New York, NY, Routledge
•
Zuboff,S.(1984).In the Age of the Smart Machine,.New York, Basic Book
•
What the research says: https://www.lkldev.ioe.ac.uk/lklinnovation/
•
Potential
of
Big
Data
and
Learning
http://modellingteachingcosts.wordpress.com/learning-analytics/
•
Potential of Learning Analsytics: https://www.lkldev.ioe.ac.uk/lklinnovation/big-dataand-learning-analytics/
•
Building Community Knowledge:
http://buildingcommunityknowledge.wordpress.com/about/
•
Modelling Teaching Costs Session:
•
http://modellingteachingcosts.wordpress.com/
Analytics: