Model for Rapid, Large-Scale Development
of
Learning Objects in Multiple Domains
IEEE International Conference on Technology for Education (T4E)
IIT Madras, Chennai
July 14-16, 2011
--- Gargi Banerjee*
Sahana Murthy*
*Interdisciplinary Program in Educational Technology,
IIT Bombay, Powai.
Introduction
Problem Statement
Solutions
Results
11Slide
Conclusion
e1
What are Learning Objects (LOs)?
• Operational definition adopted – L’Allier’s Definition, since it has
education as its focus.
• Definition :“The smallest, independent structural experience that
contains an objective, a learning activity and an assessment.”
[L’Allier,1997]
Sample LO
Introduction
Problem Statement
Solutions
Results
Conclusion
22Slide
e2
What are Learning Objects (LOs)?
• Operational definition adopted – L’Allier’s Definition, since it has
education as its focus.
• Definition :“The smallest, independent structural experience that
contains an objective, a learning activity and an assessment.”
[L’Allier,1997]
Sample LO
Introduction
Problem Statement
Solutions
Results
Conclusion
33Slide
e3
What are Learning Objects (LOs)?
• Operational definition adopted – L’Allier’s Definition, since it has
education as its focus.
• Definition :“The smallest, independent structural experience that
contains an objective, a learning activity and an assessment.”
[L’Allier,1997]
Sample LO
Introduction
Problem Statement
Solutions
Results
Conclusion
44Slide
e4
What are Learning Objects (LOs)?
• Operational definition adopted – L’Allier’s Definition, since it has
education as its focus.
• Definition :“The smallest, independent structural experience that
contains an objective, a learning activity and an assessment.”
[L’Allier,1997]
Sample LO
Introduction
Problem Statement
Solutions
Results
Conclusion
55Slide
e5
What are Learning Objects (LOs)?
• Operational definition adopted – L’Allier’s Definition, since it has
education as its focus.
• Definition :“The smallest, independent structural experience that
contains an objective, a learning activity and an assessment.”
[L’Allier,1997]
Sample LO
Introduction
Problem Statement
Solutions
Results
Conclusion
66Slide
e6
Why Learning Objects (LOs?)
• Learning objects (LO) have become valuable educational resource
a. Fit into different instructional settings
b. Ideal for spreading higher education in India
•Learning Object repositories (LORs) have increased rapidly in recent times
Introduction
Problem Statement
Solutions
Results
Conclusion
77Slide
e7
GOALS OF PROJECT OSCAR (IIT Bombay)
• To create a repository of good quality open-source courseware of LOs
• Educational Level targeted – Tertiary Level
• Multiple Domains – Varied domains like Bioscience, Civil Engg., Chemistry,
Earth Sc., Electrical Engg., Physics etc.
• High production rate – 100 LOs in 1 year
• Total of 300 LOs to be produced in 3 years
• A ‘National Mission on Education through ICT’ (NMEICT) initiative
• Funded by Ministry of Human Resources, GOI
Introduction
Problem Statement
Solutions
Results
Conclusion
88Slide
e8
How are LOs Produced ?
Pedagogy expert
SME
Interface
Designer
Coders/Animators
TYPICAL LO PRODUCTION PROCESS
• Synchronous models studied – TEAL*1,MIDOA*2, PhET*3
•Good quality LOs produced but in restricted domains
• Production Rate - 100 LOs in 4-5 years (approx.)
1 J. Belcher, “Studio Physics at MIT”. 2001, Available at: http://web.mit.edu/physics/news/physicsatmit/physicsatmit_01_teal.pdf
2 S. Arturo Barajas, A. Jaime Muñoz, R. Francisco J Álvarez., and G. Alma R.García, “Developing Large Scale Learning Objects for Software Engineering Process
Model through MIDOA Model”, Mexican International Conference on Computer Science, September 2009
3 W. K Adams., S. Reid, R. LeMaster, S.B. McKagan, K.K. Perkins, C.E. Wieman, “A study of educational simulations Part I – Engagement and Learning”, Journal
of Interactive Learning Research., 19(3). 397-419, 2008
Introduction
Problem Statement
Solutions
Results
99Slide
Conclusion
e9
Problems faced in adopting a Synchronous Model
• Scaling along numbers -Working simultaneously in
multiple domains
Team size increased
Team geographically
dispersed
• Scaling along domains -a. Finding faculty in each domain willing to
devote so much time
b. Developers’ lack of domain knowledge
aggravated ‘distance in communication’
problem
Synchronous
discussion
hampered
Conclusion :
• Synchronous model will not work, if LOs are to be developed:
a. in parallel in multiple domains
b. when there is a time constraint to production of good quality LOs
c. when a large development team is involved
Introduction
Problem Statement
Solutions
Results
1010Slide
Conclusion
e10
Broad Research Questions
1. What process to be adopted to create LOs in multiple domains at the
tertiary level of education while maintaining high quality and high
production rate?
2. Can an asynchronous, distributed development model replicate the
efficiency of synchronous development model?
Introduction
Problem Statement
Solutions
Results
1111Slide
Conclusion
e11
Production Process Adopted
Propose
concept
Review
proposal
• Concept
Proposer
Write
IDD ppt
• Reviewer
Review
IDD
Stage 1
Introduction
• IDD writer
Develop
Design
• Reviewer
Review
Design
Stage 2
Problem Statement
• Designer
• Reviewer
Stage 3
Solutions
Results
Develop
animation
• Developer
Review
animation
• Reviewer
Stage 4
1212Slide
Conclusion
e12
Production Process Adopted
• Concept
Proposer
Propose
concept
Review
proposal
Write
IDD ppt
• Reviewer
Review
IDD
Stage 1
•
• IDD writer
Develop
Design
• Reviewer
Review
Design
Stage 2
• Designer
• Reviewer
Stage 3
Develop
animation
• Developer
Review
animation
• Reviewer
Stage 4
Various asynchronous models of LO development tried A. Online Distributed Model
C. Fully Outsourced Model
B. Individual Faculty Model
D. Domain Owner
• In each of the models tried, the variable changed was : who was
responsible for a specific task
Introduction
Problem Statement
Solutions
Results
1313Slide
Conclusion
e13
A. Online Distributed Model
Process –
Propose
concept
Review
proposal
• Concept
Proposer =
Online
community
• Reviewer
= OSCAR
Write
IDD ppt
Review
IDD
Stage 1
• IDD writer
= Online
community
• Reviewer =
IITB Faculty
Stage 2
Develop
Design
Review
Design
• Designer =
OSCAR
• Reviewer =
OSCAR
Visual Design
Expert
Stage 3
Develop
animation
• Developer
= OSCAR
Review
animation
• Reviewer =
IITB Faculty
Stage 4
Advantages –
•
LOs developed by a community of teachers and learners.
Limitations –
•
Content quality of the IDD biggest problem in this model.
•
Challenging to maintain communication channels between the developer, reviewer and
the IDD writer (the person who originally proposed the idea)
Lesson Learnt •
Only if developer knew the domain could LOs under this model get developed
Introduction
Problem Statement
Solutions
Results
1414Slide
Conclusion
e14
B. Individual Faculty Model
Process –
• Concept
Proposer =
IITB Faculty
Propose
concept
Write
IDD ppt
Review
IDD
Stage 1
• IDD writer
= IITB
Student
Develop
Design
• Designer =
OSCAR
• Reviewer =
IITB Faculty
Review
Design
• Reviewer =
OSCAR
Visual Design
Expert
Stage 2
Develop
animation
• Developer
= OSCAR
Review
animation
• Reviewer =
IITB Faculty
Stage 3
Stage 4
Advantages –
•
Content quality of LOs ensured right from the beginning.
Limitations –
•
•
•
Difficult to map a set of LOs to a course
Content review took a very long time due to faculty time constraint
Communication gap between the faculty SME and the developer
Lesson Learnt:
•
Importance of standardizing instructions in asynchronous mode to replicate
effectiveness of face-to-face interactions.
Introduction
Problem Statement
Solutions
Results
1515Slide
Conclusion
e15
C. Fully Outsourced Model
Process –
Propose
concept
Review
proposal
• Concept
Proposer =
IITB Faculty
• Reviewer
= OSCAR
Write
IDD ppt
Review
IDD
Stage 1
• IDD writer
=Vendor
SME
• Reviewer =
IITB Faculty
Stage 2
Develop
Design
Review
Design
• Designer =
OSCAR
• Reviewer =
OSCAR
Visual Design
Expert
Develop
animation
• Developer
= Vendor
Review
animation
• Reviewer =
IITB Faculty
Stage 3
Stage 4
Advantages –
• Production cost low
• LO coding time reduced
Limitations –
• LOs dispersed over various content areas, not coursewise
• Gap in expected content standards between IITB SME and vendor SME.
• Overall production time increased
• Review process of IDD very lengthy
• Vendor was constantly required to adapt to different expectations.
Lessons Learnt - Content writing cannot be outsourced
Introduction
Problem Statement
Solutions
Results
1616Slide
Conclusion
e16
D. Domain Owner Model
Process –
Propose
concept
• Concept
Proposer
= IITB
Faculty
Review
proposal
Write
IDD ppt
Review
IDD
Stage 1
• IDD writer
= IITB
Student
• Reviewer =
IITB Faculty
Develop
Design
Review
Design
Stage 2
• Designer =
OSCAR
• Reviewer =
OSCAR
Visual Design
Expert
Stage 3
Develop
animation
• Developer
= Vendor
Review
animation
• Reviewer =
IITB Faculty
Stage 4
Advantages –
• Review time for both IDD and LO reduced
• Instructional Designing training given to ID writers.
Limitations –
•
Time lag between completion of the content generation phase and start of the LO
coding phase
Introduction
Problem Statement
Solutions
Results
1717Slide
Conclusion
e17
D. Detailed Domain Owner Process
SME
Validate concept
proposal
Concept
Proposal
Stage
Review of concept
proposal
OSCAR
Instructional Design
Document
Validate requirement
Validate pedagogy
IITB students
Pedagogy
Instructional
Designer
Detailed
Design
Verify design
Designer
Pedagogy
review
IITB Faculty
Pedagogy expert + IDD
writer
Design
Review
Design Reviewer
Validate LO
LO
Development
Vendor
Developer
IDD Review
LO Review
IITB Faculty
Final LO
Released
Index:
SME = Subject Matter expert, can be a faculty or a post-graduate/doctoral student;
SB= Storyboard/IDD; LO = Learning Object
Introduction
Problem Statement
Solutions
Results
1818Slide
Conclusion
e18
D. Domain Owner
Features:
1. Domain Owner model proved most successful within the constraints of
cost, schedule and quality
2. Content creation done in-house, by faculty and students at our Institute
3. IDD development outsourced to animation companies
4. Vendor did not have to contend with different SME teams
5. Mean production time fell to 0.375 months for each LO without affecting
quality [Table 1]
1919Slide
e19
Results: Comparative Analysis Of Different
Asynchronous Model
Table 1: Development Data for all the models
Model
No. of
IDDs
submitted
per month
No. of
approve
d IDDs
submitte
d/month
Mean
Time
taken
for
creating
1
approve
d IDD
[month]
Mean Time
taken to
animate 1
approved
IDD [month]
Mean Time
taken for 1
entire LO =
approved
IDD + LO
development
[month]
Mean
Cost of
LO
creation
(Rs.) / LO
Online
0.3
0
∞
-
-
Individual
Faculty
2
2
0.5
3mos/6LOs
= 0.5mos
1 months
61,358/-
Fully
Outsourced
6
1
1
4mos/20LOs
= 0.2mo
1.2months
24,600/-
Domain
Owner
4
4
0.25
2mos/16LOs
= 0.125mos
0.375months
35,530/-
-
Index: mos = months
Introduction
Problem Statement
Solutions
Results
2020Slide
Conclusion
e20
Results: Comparative Analysis Of Different
Asynchronous Model
Introduction
Problem Statement
Solutions
Results
2121Slide
Conclusion
e21
CONCLUSION
• Domain Owner model seeks to harness the expertise of both academia and
industry to create quality LOs
• Content creation part of the LO production life-cycle is the most crucial
•Social Impact of this model is enormous –
a. OSCAR LOR is able to make LOs from advanced Science and
Engineering domains accessible to students in every corner of India.
b. Students and research scholars of our Institute, many of whom are
faculty of the future, become aware of pedagogical issues
Introduction
Problem Statement
Solutions
Results
2222Slide
Conclusion
e22
ACKNOWLEDGEMENT
• The authors would like to thank the following organizations
and people for their contribution, help and support:
 NMEICT , Ministry of Human Resources & Development,
Govt. of India
 IIT Bombay
 Project OSCAR team : PIs, Project Managers, ID writers,
Designers, Animators
 IIT Bombay faculty
 All Domain Owners in Project OSCAR
2323Slide
e23
THANK YOU !
2424Slide
e24