Professional Learning through
Massive Open Online Courses
Allison Littlejohn, Colin Milligan
Caledonian Academy, Glasgow Caledonian University Glasgow, Scotland
Obiageli Ukadike
Harvard Medical School – Harvard Catalyst, Boston MA
@allisonl @cdmilligan
Technology-enhanced Professional Learning
Littlejohn & Margaryan, 2013
page 2
Technology-enhanced Professional Learning
Personalised
Social
Continual
Self-regulated
Redefined relationships
Littlejohn & Margaryan, 2013
page 3
Social cognitive model of self- regulation
(Zimmerman, 2000)
Self-regulation is the ‘self-generated thoughts, feelings and actions
that are planned and cyclically adapted to the attainment of
personal goals’
Forethought
Performance
Task analysis
Goal setting
Strategic planning
Self-motivation belief
Self-efficacy
Task interest/value
Self-control
Self-judgement
Task strategies
Self-evaluation
Elaboration
Self-reaction
Critical Thinking
Self-satisfaction/affect
Help seeking
Interest enhancement
Forethought
Selfreflection
page 4
Performance
Self-reflection
Learner
Learning
strategy
Learning
behaviour
Learning
opportunities
Learning
tools
page 5
What’s known about Professional Learning in MOOCs
Tensions across autonomy- diversity, openess- connectedness in
MOOCs
Mackness, Mak & Williams (2010)
Critical/ digital literacies in MOOCs
Fini et al (2009), Kop et al (2011)
Self-regulated learning behaviours in a MOOC
Milligan, Littlejohn & Margaryan (2013)
page 6
What’s known about Professional Learning in MOOCs
Project provides new insight into:
• Learning behaviours in MOOCs while self-regulating
• MOOC environment design recommendations to encourage the
development and emergence of SRL behaviours.
Relevant for professional learning, though applicable to a range of
settings.
Outputs aimed to impact future research and development
page 7
Research Questions
• RQ: What self-regulated learning strategies and behaviours do
professionals adopt in MOOCs?
• RQ: How are Massive Open Online Courses currently designed
to support self-regulated learning?
• RQ: How can MOOCs be designed to encourage professionals
to self-regulate their learning?
Context:
• MOOC for healthcare professionals (edX: HarvardX)
• Findings and outputs tested in other health MOOCs
(FutureLearn: Kings College London)
page 8
Research Context: edX ‘Fundamentals of Clinical Trials’
• Harvard Medical School, Harvard School of Public Health,
Harvard Catalyst.
• Delivered via edX Oct 2013-Feb2014
https://www.edx.org/course/harvardx/harvardx-hsph-hms214x-fundamentals-941
• 24,000 registered, 400+ participants in study, contacted by
course announcement in week 5 (of 14).
• Provides an introduction to the scientific, statistical, and ethical
aspects of clinical trials research.
• Participants assumed to have a background in biostatistics and
epidemiology.
page 9
Fundamentals of Clinical Trials
Video, with
commentary...
page 10
Fundamentals of Clinical Trials
Follow the
textbook link to
read the ebook
online…
page 11
Fundamentals of Clinical Trials
Discussion
forum,
encourages indepth discussion
page 12
Fundamentals of Clinical Trials
MCQ assessment
items.
Limited number
of attempts
Scores are
recorded in the
progress tab.
page 13
RQ: What self-regulated learning strategies and
behaviours do professionals adopt in MOOCs?
Collect and analyse learner data form professionals learning in a MOOC.
• Survey of SRL profiles to collect data of overall population.
• Learner interviews to explore impact of course on practice.
• Data analysis and synthesis of quant. and qual. data.
(leading to)
• Use Cases of learner behaviours providing a narrative description of
learner behaviour patterns observed.
page 14
RQ: What self-regulated learning strategies and
behaviours do professionals adopt?
• SRL Profile Survey Instrument:
http://figshare.com/articles/SRLMQ/866774
• 42 items measuring 11 sub-processes of SRL.
• Based on a validated instrument exploring SRL behaviours in
professional learning.
• Three open ended questions exploring:
• motivation,
• impact,
• interaction
• Administered via SurveyMonkey
page 15
Sample questions (measured on 5pt Likert scale)
• My past experiences prepare me well for new learning
challenges. (self-efficacy)
• I try to apply my previous experience when learning.
(elaboration)
• I ask myself how what I am learning is related to what I already
know. (task strategies)
• I ask others for more information when I need it. (help-seeking)
• I try to understand how what I have learned impacts my
work/practice. (self-satisfaction)
page 16
Snapshot of Sample: Demographics
Continent
Countries
Participants
Number
Countries
South America
25
6
US – United States
64
North America
82
11
IN - India
38
Europe
109
23
ES - Spain
20
Africa
36
15
EG - Egypt
13
Asia
87
20
BR - Brazil
12
Oceania
8
2
UK – United Kingdom
12
Not stated
29
-
376
77
page 17
Snapshot of Sample: Demographics
Age profile: 60% under 35
Gender F:M 52:48
Education: 60% have at least a Masters level qualification
Intend to Gain Completion Certificate: 96%
Participated in a MOOC before: 52%
page 18
SRL – Profile Score
Those who identified as healthcare professionals were selected
and split into three groups based on SRL profile score.
SRL-L: ave. 125
50
45
SRL-M: ave. 153
40
SRL-H: ave .180
Frequency
35
30
25
20
15
10
5
0
SRL-Score
page 19
Snapshot of Sample: Roles
50% currently employed as a healthcare professionals
Clinician (Physician, MD, specific titles)
Clinical research (specialist job titles)
Researcher/lecturer
Pharmacist
Corporate/ non scientific roles
Other healthcare (Nurse, Physical Therapist etc.)
Dentist
Other
Total
page 20
62
36
27
18
16
12
5
1
177
Q1 What motivated you to take this course?
Interest
(responses indicated a generic expression of interest)
Topic
(referred to the topic of the course but no link to current or future role)
Career
(linked participation to their future career)
Professional
(linked to current practice, or what they hoped to get out of the course)
Harvard
(referred explicitly to certification, or prestige of provider)
MOOCs
Total
Low
3
High
6
12
4
5
9
6
14
11
3
37
1
37
Low and High SRL groups seem to differ in motivation:
• Low attracted by prestige of HarvardX Brand or by topic.
• High cite links to career or professional practice
page 21
Q2 Please describe how you expect studying on this course
to contribute to your work or professional practice (or other
studies)?
Interest
(doing this for interest rather than to gain specific knowledge)
Knowledge
(comments highlight topic focus but do not relate directly to current or future role)
Practice
(focused specifically on current role)
Career
(focused on future role)
Total
Low High
4
0
Little difference between the two groups:
• Only Low-SRL described participation as ‘for interest only’
page 22
8
12
18
16
6
7
36
35
Q3 Please describe your experience of interacting with
other learners in the discussion forum.
What went well? What was challenging?
Positive
Negative
 Language /technical
 Frustrated
 Overwhelmed
Not used
 Not interested
 No time
Low
13
6
4
1
2
13
6
7
Broadly similar though:
• Low-SRL may be less likely to prioritise time,
• Only low-SRL learners were ‘overwhelmed’.
page 23
High
15
5
2
3
0
11
8
3
Semi-structured interviews (begin tomorrow).
• Minimum 30 (up to 70) participants (from two groups: low-SRL and
high-SRL, all professionals) contacted for interview.
• Research participants asked to reflect on the impact that the MOOC
has had/is likely to have on their professional practice.
• SS Interview script:
http://figshare.com/articles/PL_MOOC_Semi_structured_interview_script/866773
page 24
Sample questions
• Context: Do you expect to integrate what you have learned in the
MOOC into your professional practice?
Can you give an example? [PROBE: How?]
• Strategies: Did you share anything with anyone else either formally
or informally (e.g. through informal discussions, postings on blogs,
contribution to discussion forums etc.)
[PROBE: Who?]
• Self-evaluation: Did you talk to anyone in your professional network
to discuss the value of the MOOC?
page 25
Semi-structured interviews
• Basis for use case narratives of learning behaviours in a MOOC
• Questions (aligned with SRL-sub-processes) explore:
• interaction with others, within and beyond course (e.g. with
professional network)
• use of tools to support interaction with course content and to support
learning (using 4c’s: consume, connect, create and contribute)
• explore impact on professional practice.
• First interviews scheduled for 6/12/13
• Continue throughout December
• Analysis in January 2014
page 26
RQ: How are Massive Open Online Courses currently
designed to support self-regulated learning?
Review current research in SRL and develop a tool to allow mapping of
MOOC designs and SRL behaviours.
• Desk research of SRL to inform Design Mapping Tool .
• Course Design document review.
• edX/HarvardX Design team interview.
• Data analysis and synthesis mapping design to SRL sub-processes.
(leading to)
• Design mapping tool based on current SRL literature.
• MOOC design strategy report.
page 27
RQ: How are Massive Open Online Courses currently
designed to support self-regulated learning?
• Explore learner behaviour AND MOOC design through SRL lens
• Complement the learner data with data about the design of the
course (at strategic, platform and course level)
• Environments can foster the development of SRL Skills
• Learner’s ability to self-regulate impacts their performance
• Survey research evidence and map to the SRL sub-processes.
• Develop questions for MOOC providers, platform developers,
course designers and teaching assistants to build picture of
individual MOOCs
page 28
Summary of research evidence
What interventions might you
make in an environment to
promote SRL.













What SRL-related learner
behaviours are expressed in
online environments (and
correlated with better academic
performance):
page 29







(pre-training/presence of tutor) (Azevedo et al, 2004, 2005)
F: Goal-setting tools (Chang et al 2013) (also flexible pathways through content)
F: Planning tools (experimental plans) (White and Frederiksen, 1998)
F: Planning tools (cues, hints) (Manlove et al 2008)
P: Scaffold interaction (Cho & Kim, 2013)
P: Require interaction (Cho & Kim, 2013)
P: Feedback mechanisms (Aleven et al, 2006) (Wang, 2011)
P: Note-taking and Highlighting tools (Proske, Narciss, and Koerndle, 2007) (Kauffman,
2004)
P: Cues to take structured notes (Kauffman, 2004)
P/SR: Diary tools (Arsal, 2010; Neber &Schommer-Atkins, 2002; Schmitz & Weise, 2006;
Winne, 2005)
SR: Inline prompts to monitor/reflect on learning (Kauffman, 2004)
SR Explicit meta-cognitive tasks (Delfino, Dettori & Persico, 2011 ; Kramarski & Mizrachi,
2013)
P: Tasks structures encouraging participants to take control (Dettori, Giannetti & Persico,
2005)
F: Goal setting (Chang et al, 2013)
F: Planning (Azevedo et al, 2004)
F: Self-efficacy (Wang and Lin, 2007) (Yang et al, 2006) (Cho and Johanssen, 2009)
F: Goal orientation (Chang, 2007)
P: Monitoring (Greene and Azevedo, 2009)
P: Time management: (Hu and Gramling, 2009; Song et al, 2004)
P: Help-seeking (Azevedo et al, 2004; Vighnarajah et al, 2009)
Design Mapping Tool
SRL Sub-processes
Evidence
page 30
Planning
Goal-setting
Self-efficacy
Taskinterest/value
Help-seeking
Task strategies
Elaboration
Self-evaluation
Self-satisfaction
Domains
Environment
Design
Recommendations
RQ: How can MOOCs be designed to encourage
professionals to self-regulate their learning?
Synthesise results as recommendations and scientific outputs.
• Data analysis of data collected in first two phases into a set of
recommendations.
• Recommendations review with additional professional learning
MOOC.
(leading to)
• Revised MOOC design recommendations for MOOC community.
• All instruments, datasets and findings from the study (eg IRRODL
article)
page 31
Current and Next steps
• RQ2 Conduct interviews and analyse learner behaviour
• RQ1 Collect MOOC design data using mapping tool
• RQ3 Develop recommendations for MOOC design
• Refine tools and recommendations in a different MOOC
page 32
Current and Next steps
Provide new insight of use to the research & development
communities:
• Learning behaviours in MOOCs
• MOOC environment design recommendations
page 33
Professional Learning through
Massive Open Online Courses
Allison Littlejohn, Colin Milligan
Caledonian Academy, Glasgow Caledonian University Glasgow, Scotland
Obiageli Ukadike
Harvard Medical School – Harvard Catalyst, Boston MA
http://gcu.ac.uk/academy/pl-mooc/
@allisonl @cdmilligan
Bibliography
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Barnard, L., Paton, V., & Lan, W. (2008). Online self-regulatory learning behaviours as a mediator in the relationship between
online course perceptions and achievements. International Review of Open and Distance Learning, 9 (2), 1-11.
Castells, M. (1996). The information age: economy, society and culture: The rise of the networked society. Oxford, UK: Blackwell.
Bernacki, M.L., Aguilar, A.C., and Byrnes, J.P. (2011) Self-regulated learning and technology-enhanced learning environments: an
opportunity-propensity analysis. In: G. Dettori & D. Persico (Eds.) Fostering Self-regulated learning through ICT. pp. 364-379. IGI
Global: Hershey, PA,
Cho, M-H., & Kim, B.J. (2013). Students’ self-regulation for interaction with others in online learning environments. The Internet
and higher Education, 17, 69-75.
Dettori, G.,& Persico, D. (2009) Supporting self-regulated learning with ICT, in A. Cartelli and M. Palma (eds.) Encyclopaedia of
ICT, pp 735-741. IGI Global, Hershey, PA
Daniel, J. (2012). Making Sense of MOOCs: Musings in a Maze of Myth, Paradox and Possibility. Journal of Interactive Media In
Education, 3(0). Retrieved August 8, 2013, from http://www-jime.open.ac.uk/jime/article/view/2012-18
Downes, S. (2009). Connectivist dynamics in communities. Retrieved 26 February 2013
from http://halfanhour.blogspot.co.uk/2009/02/connectivist-dynamics-in-communities.html
Engeström, R. (2009). Who Is Acting in an Activity System? In A. Sannino, H. Daniels, & K. D. Gutiérrez (Eds.), Learning and
Expanding with Activity Theory, pp. 257-273. New York, NY: Cambridge University Press.
Eraut, M. (2000). Non-formal learning and tacit knowledge in professional work.British Journal of Educational Psychology, 70, 113136.
Fini, A. (2009). The technological dimension of a Massive Open Online Course: the case of the CCK08 course tools. The
International Review of Research in Open and Distance Learning, 10(5), 74-99.
Kop, R. (2011). The challenges to connectivist learning on open online networks: learning experiences during a Massive Open
Online Course. The International Review of Research in Open and Distance Learning, 12 (3), 19-38.
Kop, R., & Fournier, H. (2011). New dimensions to self-directed learning in an open networked learning environment. International
J. of Self-Directed Learning,7 (2), 1-18.
Lefrere, P. (2007). “Business success - the special contribution of self-regulated learning.” In J. Beishuizen, R. Carniero & K.
Steffens (Eds.) Self-regulated Learning in Technology Enhanced Learning Environments: Individual Learning and Communities of
Learners. pp. 49-53. Shaker Verlag: Aachen.
Littlejohn, A., Milligan, C., & Margaryan, A. (2011). Collective learning in the workplace: Important knowledge sharing
behaviours. International Journal of Advanced Corporate Learning, 4 (4) 26-31.
page 35
Bibliography
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Littlejohn, A., Milligan, C., Margaryan, A (2012),"Charting collective knowledge: supporting
self-regulated learning in the workplace", Journal of Workplace Learning, 24(3).226-238.
Littlejohn, A., & Margaryan, A. (2013). Technology Enhanced Professional Learning: mapping out a new domain. Chapter 1 in
Littlejohn, A., & Margaryan, A. (Eds.). Technology-enhanced professional learning: Processes, practices and tools. London, Routledge.
Liyanagunawardena, T. R., Adams, A. A., & Williams, S.A. (2013). MOOCs: A systematic study of the published literature 20082012, International Review of Research in Open and Distance Learning, 14 (3) 202-227.
Mackness, J., Mak, S., & Williams, R. (2010). The ideals and reality of participating in a MOOC. In L. Dirckinck-Holmfeld, V. Hodgson,
C. Jones, M. de Laat, D. McConnell, & T. Ryberg. (Eds.). Proceedings of the 7th International Conference on Networked Learning
2010. pp. 266-275. University of Lancaster, Lancaster.
Milligan, C. Littlejohn, A., & Margaryan, A. (2013). Patterns of Engagement in Connectivist MOOCs. Journal of Online Learning &
Teaching 9 (2) 149-159. http://jolt.merlot.org/vol9no2/milligan_0613.pdf
Milligan, C., Margaryan, A., & Littlejohn, A. (2013). Goal-setting behaviour in massive open online courses, EARLI 2013
Conference. Munich Germany, 27-31 August, 2013. https://t.co/KsO0uMR5Fv
Pintrich, P. R., Smith, D., García, T., & McKeachie, W. (1991). A manual for the use of the Motivated Strategies for Learning
Questionnaire (MSLQ). Ann Arbor. Michigan.
Schunk, D. H. (2001). Self-regulation through goal-setting, ERIC Digest. Retrieved August 8, 2013,
fromhttp://www.counseling.org/Resources/Library/ERIC%20Digests/2001-08.pdf
Siadaty, M., Jovanović, J., & Gašević, D. (2013). The social semantic web and workplace learning. Chapter 12 in Littlejohn, A., &
Margaryan, A. (Eds.).Technology-enhanced professional learning: Processes, practices and tools. (forthcoming) London, Routledge.
Siemens, G. (2005). Connectivism: a learning theory for the digital age. International Journal of Instructional Technology and Distance
Learning, 2(1), 3-10.
Tynjälä, P. (2008). Perspectives into learning in the workplace. Educational Research Review, 3, 130-154.
Veen, W., van Staalduinen, J-P. & Hennis, T. (2011). Informal self-regulated learning in corporate organizations. In: G. Dettori & D.
Persico (Eds.) Fostering Self-regulated learning through ICT. pp. 364-379. IGI Global: Hershey, PA,
Zimmerman, B. J. (2000). Attaining Self-regulation: A Social Cognitive Perspective. In M. Boekaerts, P.R. Pintrich, & M. Zeidner
(Eds.), Handbook of self-regulation, San Diego, Academic Press.
page 36
Design mapping tool example questions: Help-seeking
Questions
- Is interaction with other learners seen as
necessary/essential for course participation? (strategic)
- Are tools/structures provided to encourage interaction?
(platform)
- Are tools /structures provided that encourage interaction
around specific pieces of content (e.g. collaborative
annotation?) (platform)
- Are learners encouraged to interact with each other, and
tutors as they learn (course dev.)
- Are learners expected to make contact with other learners
to discuss course content (course dev.)
- Are learners expected to make contact with tutors
(teaching assistants) to discuss course content? (course
dev)
- Have you observed learners making contact with each
other in the course discussion fora (teaching assistant).
page 37
Research Evidence
- Help-seeking correlates with
increased academic performance
(Azevedo et al, 2004; Vighnarajah et
al, 2009)
- Scaffolding interaction promotes
SRL (Cho & Kim, 2013)
- Requiring interaction promotes
SRL (Cho & Kim, 2013)