Teaching as a design science:
Designing and assessing the effectiveness
of the pedagogy in learning technologies
Diana Laurillard
London Knowledge Lab
Institute of Education
eLearning Forum Asia 2013
Hong Kong Baptist University
29-31 May 2013
The Challenges to Higher Education
The issues
• Global demand for HE
• The aims of HE
• The roles of TEL
• Modelling learning technologies
• Teachers as innovators
The policy context
• Teachers to have the capacity to plan and deliver ICT-enriched learning
experiences for students to become self-directed and collaborative
learners... (Singapore ICT Masterplan, 2008)
• Teacher capacities will be developed in instructional design, selection and
critical evaluation of digital content, and strategies for effective use of
digital content to enhance student learning. (India Government, 2009)
• Professional educators connected by technology to empower, and inspire
effective teaching (US Plan 2010)
• … greater prioritisation of teaching partnerships between technologists,
learning support specialists and academics, and an end to the ‘not invented
here’ syndrome… Good practice must also be shared. (HEFCE OLTF, 2011)
• Professional development is needed for all school staff to support the
process of ICT integration in schools (UNESCO, ICT in Primary, 2012)
The global demand for HE
By 2025, the global demand for higher education will double
from 100m to ~200m per year, mostly from emerging
economies (NAFSA 2010)
The new UNESCO goals for education:
• Every child completes a full 9 years of
free basic education …
• Post-basic education expanded to
meet needs for knowledge and skills …
(Draft for UNESCO post 2015 goals)
- Implying significant teacher training needs for HE
25:1 student:staff ratio = 4m new teachers??
The aims and purpose of HE
The purposes of higher education:
• To inspire and enable individuals to develop their
capabilities to the highest…
• To increase knowledge and understanding for their
own sake…
• To serve the needs of an adaptable, sustainable
knowledge-based economy…
• To play a major role in shaping a democratic,
civilised, inclusive society (NCIHE, 1997)
How could mass HE achieve that nurturing of the
individual, while reducing the current 25:1 student:staff
ratio for student support?
An example: The Duke MOOC
Bioelectricity: A Quantitative Approach
Taught in class for over 20 years
Experimental move to a free and open MOOC
12,000 students enrolled from >100 countries
• 8 weeks long
• 97 ~6 min videos
• 22 GB of data
• 1052 files
• 18 graded exercises, including a peer-graded writing
assignment and final exam
(Duke University 2013)
The Duke MOOC
Not for undergraduates
Enrolled students
Potential
undergraduates
The Duke MOOC
Not for the faint-hearted
Comparable with
normal online u/g
courses
Basic MOOCs vs the Duke MOOC:
Comparing the learning experience
Basic: 8 weeks, providing 50 hours learning time, no teacher support
Duke: 8 weeks, providing 50 hours learning time including support:
Videos and pdfs
Quizzes
High on prep time
Wiki
Zero contact for 42 hours
Peer discussions
Peer grading
Low on prep time
Tutored discussions
High contact for 8 hours
Summative assessment
420 hours to develop materials and course design
200 hours to support ~500 students for 8 hours = 20:1 student:staff ratio
Report at http://bit.ly/ZRMbjp
Comparing teacher hours for a basic MOOC and
the Duke MOOC (48 hour course)
Total teacher time
Teacher support
time rises to 2000
hours for 5000
students.
3000
2500
2000
Duke MOOC
1500
Basic MOOC
1000
200
Prep
time
500
2000 hours
= 1 year of a tutor
for a 5 credit
course.
0
50
500
5000
= 24 FT tutors for
120 credit course.
Prep time = 420 hrs
Support time
50
500
5000
Duke MOOC
20 hrs
200 hrs
2000 hrs
Basic MOOC
0.00
0.00
0.00
Modelling the benefits and costs
• We need to understand the pedagogical
benefits and teacher time costs of online HE
• What are the new digital pedagogies that will
address the 25:1 student support conundrum?
• Who will innovate, test, and build the evidence
for what works at scale online?
TEACHERS!
Pedagogies for supporting large classes
Concealed MCQs
The virtual Keller Plan
The vicarious master class
Pyramid discussion groups
Conceal answers to question
Ask for user-constructed input
Introduce
content
Reveal multiple
answers
Self-paced
Ask user to practice
select nearest fit
Tutor-marked
test
240
individual
students produce
Tutorial
for
5
representative
Student
becomes
tutor for credit
response
to open question
students
Until
half
class and
is tutoring
thejoint
rest
Pairs
compare
produce
Questions
and guidance
represent
response
all students’ needs
Groups of 4 compare and produce
joint response and post as one of
10 responses...
6 groups of 40 students vote on
best response
Teacher receives 6 responses to
comment on
Digital pedagogies for scaling up to higher
student:staff ratios without losing quality
Concealed MCQs
Like MCQs, no teacher support
More challenging learning experience
The virtual Keller Plan
Little teacher support
Every learner receives 1-1 support
The vicarious master class One teacher guides 5 students
They represent all students’ needs
Pyramid discussion groups Teacher reacts to 6 student outputs
All students contribute, discuss, vote
How might technology help teachers?
First things first – how do students learn?
How does technology help?
Tools for teachers to build pedagogic knowledge
Teachers as design scientists…
The learner learning
Acquiring
L
L
Learner
C
C
concepts
Teacher
concepts
Inquiring
Modulate
Generate
L
L
Learner
P
P
practice
Learning through acquisition, instruction
Learning through inquiry
The learner learning
L
L
Learner
C
C
concepts
Teacher
concepts
Modulate
Modulate
Generate
Generate
Task
Feedback
L
L
Learner
P
P
practice
Learning
environment
Actions
Learning through practice with meaningful intrinsic feedback
The learner learning
Acquiring
Ideas, questions
L
L
Learner
C
C
concepts
Teacher
concepts
Peer
concepts
Ideas, questions
Inquiring
Modulate
Modulate
Modulate
Generate
Generate
Task/Feedback
Generate
Outputs
L
L
Learner
P
P
practice
Learning
environment
Actions
Peer
practice
Outputs
Learning through discussion from peers’ ideas, questions
Learning through sharing from peers’ practice
Representing types of learning
Acquiring
Teacher
concepts
Teacher
communication
Inquiring
cycle
Producing
Modulate
Peer
communication
Discussing
cycle
Peer
concepts
Peer
modelling
Sharing
cycle
Peer
practice
Modulate
Generate
Generate
Learning
environment
L
L
Learner
C
C
concepts
Teacher
modelling
Practising
cycle
L
L
Learner
P
P
practice
The teacher needs to design for all these types of learning
Books,
lectures,
Libraries,
journals,
papers,
websites,
repositories,
videos,
AR, podcasts…
databases,
Web…
Tutorials, seminars,
small groups, online
forums, social media…
Deploying learning technologies
Book… Podcast
Teacher
concepts
Teacher
communication
Library… Web
cycle
Essay… Model
Modulate
Peer
Tutorial…
communication
Facebook
cycle
Peer
concepts
Peer
modelling
Project…
VLE
cycle
Peer
practice
Modulate
Generate
Generate
Learning
environment
L
L
Learner
C
C
concepts
Teacher
Lab…
modelling
Simulatio
cycle
n
L
L
Learner
P
P
practice
Labs, exercises, problems,
Essays, designs, performance,
Project groups,
The
teacher
needs to
use all
types
of learning
projects,
serious
programs,
games,
videos,
ppts,
teamwork,
digital
Google
technology
within
Conversational
Framework
models,
simulations…
assets,the
models,
e-portfolios…
groups,
wikis, VLEs…
Collecting learning analytics
What accessed in
what sequence
Questions asked
Book… Podcast
Teacher
concepts
Teacher
communication
Library… Web
cycle
Essay… Model
Modulate
Peer
Tutorial…
communication
Facebook
cycle
Peer
concepts
Peer
modelling
Project…
VLE
cycle
Peer
practice
Modulate
Generate
Generate
Learning
environment
L
L
Learner
C
C
concepts
Social interaction
patterns
Teacher
Lab…
modelling
Simulatio
cycle
n
Tracked inputs,
reaction times,
sequence
L
L
Learner
P
P
practice
Analysis of essay content,
quiz scores, game scores,
accuracy of model
Tracked group
outputs, peer
assessment
Forms of TEL/online learning activities
Learning activities for online courses
• Guided TEL resources (model, AR)
• Access to expositions (lecture videos)
• Automated grading (MCQs, models)
• Readings (pdfs)
• Guided collaboration activities (wiki, AR)
• Peer group discussion (forum)
• Peer grading against criteria (share)
• Tutored discussion (forum)
• Tutor feedback (assignments)
• Tutor-based assessment (exams)
What are effective ways
of combining and
sequencing these
activities for learning and
formative assessment,
that will help students
achieve the intended
outcomes?
Tools for teachers as learning designers
Teachers as designers need the tools for innovation
To find or
create new
ideas
To collect
learning
analytics
Adopt
Redesign
Adapt
Analyse
Test
Publish
Creating knowledge about effective online
pedagogies for specific learning outcomes
Laurillard, D. (2012). Teaching as a Design Science: Building Pedagogical Patterns for Learning and Technology. New York and London: Routledge.
Tools for teachers as learning designers
Tools for teachers as designers - PPC
A library of
patterns to
inspect
Tools for teachers as designers - PPC
Each pattern is designed to
deliver a specific type of
learning outcome
Colour-coded text
identifies content
parameters
Black text expresses
pedagogy design
Transferring the pedagogy across topics - PPC
The Pedagogical Patterns Collector
Transferring the pedagogy across topics - PPC
The Pedagogical Patterns Collector
Transferring the pedagogy across topics - PPC
Teacher adopts and adapts a design
Add link to an OER,
e.g. a digital tool for
practice
Specify duration of
the activity in mins
Read, Watch, Listen
Inquire
Discuss
Practice
Share
Produce
Adjust the type of
learning activity.
Edit the instructions.
Export to
Word
[Moodle]
Check the feedback
on the overall
distribution of
learning activity
Represent the
teacher as
present or not
Fully specifies the pedagogy for others to adopt, adapt, test, and share
Design for Med students in PPC
Explain how to optimise the inputs to a patient simulator to
achieve the ideal blood pressure
With your partner select different inputs to the patient
simulator – can you improve on your previous results?
Export to Moodle for Med students
Re-design for Business students in PPC
Explain how to optimise the inputs to a patient simulator to
achieve
the ideal
blood
pressure
Explain
how to
optimise
the inputs to a business model
to achieve the optimal cash flow
With your partner select different inputs to the
With your partner select different inputs to the patient
business model – can you improve on your previous
simulator – can you improve on your previous results?
results?
Export
to to
Moodle
forfor
Business
students
Export
Moodle
Med students
The process of professional
collaboration:
Find effective patterns for a learning objective
– adopt, adapt, improve – export to Moodle –
test with students – improve – test - publish
Comments on the PPC
• [The pie-chart] is one of the most useful features … it
gives a good overview of the balance between different
learning experiences
• I rarely consider how the students' time is apportioned …
it's good to be made to think about this.
• Seeing how the sessions are shaping up in such a visual
medium …. would probably make me think more
carefully about providing a mix of activities
Tools for teachers as learning designers
教学设计库
PPC 浏覽器
欢迎
这套教学设计库工具包使
教师能够分享他们好的教
学理念。其目的是帮助学
科教师了解某个特定教学
设计可以如何成功地应用
于不同的教学主题 。系統
中有教学设计样本供浏览
和编辑，你也可以从头开
始自行设计。这是由
ESRC- EPSRC资助的
TLRP-TEL研究项目之
‘教学设计支持环境’的
一项成果。
浏覽器
提供了一些通用的教学设计及其
相关的实例，你可以参考或改编
这些设计用于自己的教学设计之
上
PPC设计师
展示教学设计模板来帮助你
描述自己某节课的教学理念
（如学生准备、课堂活动和
家庭作业）
[Chinese version coming soon…]
设计师
Modelling learning experience and
teacher workload
With a computational model, we can estimate the
effects of the design decisions on quality of learning
experience AND teaching cost:
consequences for the pedagogical benefits
comparative costs of teachers’ workload
Then check actual outcomes using targeted learning
analytics
Enhancement: Modelling the pedagogic benefits
A computational representation can analyse how much of
each activity has been designed in
Categorised
learning activities
Conventional
Acquisition
Inquiry
Discussion
Practice
Production
Blended
Acquisition
Inquiry
Discussion
Practice
Production
Analysis shows more
active learning
Efficiency: Modelling teacher time
Learning activities for online courses
Preparation time (fixed costs)
• Guided TEL resources (model, AR)
• Access to expositions (lecture videos)
• Automated grading (MCQs, models)
• Readings (pdfs)
• Guided collaboration activities (wiki, AR)
• Peer group discussion (forum)
• Peer grading against criteria (share)
• Tutored discussion (forum)
• Tutor feedback (assignments)
• Tutor-based assessment (exams)
Support time (variable costs)
Modelling the teacher’s workload for increasing
student cohort size
Planning teacher time for a
new online course at IOE
1400
1200
1000
1000
800
800
600
600
400
400
200
200
0
0
Support
Preparation
Year 1 Year 2 Stable
state
Year 1 Year 2 Stable
state
40
40
40
40
80
160
Students
-£13000
-£600
£2000
-£13000
£19000
£61000
Profit/Loss
Modelling the teacher’s workload for increasing
student cohort size
From the Duke MOOC
3000
2500
Total
teacher
time in
hours
2000
Time for
Duke
MOOC
student
Basic
MOOC
support
1500
1000
500
0
50
500
5000
Time for
preparation
of
resources
Cohort size
Scaling up will never improve
the student support costs… unless…
Modelling the costs for increasing student
cohort size
… we develop some clever pedagogical
patterns that support students at better
than the 25:1 student:staff ratio
The question is – what are they, and how
do we develop and share them?
Invest in the teachers who can
innovate with learning technologies!
Further details…
tinyurl.com/ppcollector
Rethinking University Teaching:
A Conversational Framework for
the Effective Use of Learning
Technologies (Routledge, 2002)
(Chinese edition ECNU Press)
Teaching as a Design Science:
Building pedagogical patterns
for learning and
technology(Routledge, 2012)
d.laurillard@ioe.ac.uk
Teaching as a design science: SUMMARY
The global demand for HE requires investment in
pedagogic innovation to deliver high quality at scale
Technology-based pedagogic innovation must support
students at a better than 25:1 student:staff ratio
Teachers need the tools to design, test, gather the
evidence of what works, model benefits and costs
Teachers are the engine of innovation – designing,
testing, sharing their best pedagogic ideas
The Learning Designer A TLRP-TEL project
The project partners
Oxford
Liz Masterman (CoPI)
Marion Manton (CoPI)
Joanna Wild (RF)
IOE/LKL
Birkbeck/LKL
George Magooulas (CoPI)
Patricia Charlton
Dionisis Dimakopoulos
Brock Craft (RF)
Diana Laurillard (PI)
Dejan Ljubojevic (RF)
LondonMet
Tom Boyle (CoPI)
RVC
ALT
Seb Schmoller
Rachel Harris
LSE
Steve Ryan (CoPI)
Ed Whitley
Roser Pujadas (PhD Student)
Project website at www.ldse.org.uk
PPC at web.lkldev.ioe.ac.uk/PPC/live/ODC.html
Kim Whittlestone (CoPI)
Stephen May
Carrie Roder (PhD Student)