Stephen Marshall & Geoff Mitchell

• Assumptions
• The Problem
• Blackboard Transformation Framework
• An e-Learning Maturity Model
• Levels
• Structure
• Process categories
• An example category
• Testing the practices
• Questions?
University Teaching Development Centre
Victoria University of Wellington http://www.vuw.ac.nz/utdc/
2

• We are talking about e-Learning in the broadest sense
• Thoughtful incorporation of technology into teaching has a positive effect on student learning
• We are dealing with e-Learning at the institutional level
• There is no silver bullet
University Teaching Development Centre
Victoria University of Wellington http://www.vuw.ac.nz/utdc/
3

$
Well defined solutions?
University Teaching Development Centre
Victoria University of Wellington http://www.vuw.ac.nz/utdc/
4

University Teaching Development Centre
Victoria University of Wellington http://www.vuw.ac.nz/utdc/
5

From Operational to Transformative
The Educational Technology Framework
Bob Cupitt – Regional Manager
Dan McFadyen – Director of Global Services
Copyright © 2004 Blackboard Inc.

Educational Technology Framework
Exploratory Supported Strategic Mission Critical Transformative
Adaptive
Managed
Predictive
Phase I
Basic
Phase 2 Phase 3
Time
Phase 4 Phase 5
Copyright © 2004 Blackboard Inc.

Exploratory Phase
Uni-functional
One Department, One Person
Few Participants
De-Centralized with Multiple initiatives
Selection process
Features and Function Driven
Specific Individual needs
Acquired through creative financing
Transition 1: Basic
 Technology champions emerge
 Training initiatives begin
 Identification of best practices
 from “word of mouth” to collective voice
 Departmental involvement
Exploratory Supported
Phase I
Basic
Phase 2
Copyright © 2004 Blackboard Inc.

Supported Phase
Cross-departmental
Groups of participants
Low level institutional teaching/support model
Technology “Hand-Off”
Features and Function Driven
Specific Individual needs as well as group collaboration
Acquired through existing operating budgets
Supported Strategic
Managed
Transition 2: Managed
 Training programs provided regularly
 Ad-hoc help desk evolves
 Policies and procedures develop
 Consistent and significant messaging Phase 2
 Existing support staff claim additional responsibilities
Phase 3
Copyright © 2004 Blackboard Inc.

Strategic Phase
Significant mass of users and momentum
Organizational responsibilities designed
Enterprise technology driven decision
Integration with other institutional systems
Meets a variety of constituent based needs
Academic technology planning begins, committees form
Operational as well as Ad-Hoc capital budget
Strategic Mission Critical
Predictive
Transition 3
 “Always on” mentality
 Academic/Administrative collaboration defined
 Well known processes and procedures
 Implementing academic/strategic plan
 Systems integration begins
Phase 3 Phase 4
Copyright © 2004 Blackboard Inc.

Mission Critical Phase
Widespread institutional adoption, critical mass of users
Executing academic technology plan
Centralized service and support
Academic/Administrative collaboration
Institutional technology consolidation
Committed budget, executive sponsorship and extreme visibility
Mission Critical Transformative
Adaptive
Transition 4
 Widespread institutional adoption
 Key component for educational delivery
 Essential to all academic constituents
 SIS level significance Phase 4
 Top Down/Bottom Up awareness and responsibilities
Phase 5
Copyright © 2004 Blackboard Inc.

Transformative Phase
Strategic plan is key to institutional success
Integral to curriculum delivery
Centralized academic technology resource
Representative of Learning Organization
Core to the Organization, brand impact
“Off the Top” Funding
Organizational and technological transformation
Transformative
Final Stage
 Actual curriculum changes are dependent on the academic technologies
 Represents the institutional experience, and critical for learner experience
 Firmly established as critical delivery mechanism
Technology is Transparent
Phase 5
Copyright © 2004 Blackboard Inc.

Phase Activities
Planning None Departmental Centralizing Centralized Institutional
Budgetary
“Out of Pocket” Operating Operating Capital “Off the Top”
Technology Individual Departmental Organizational Institutional Extensible
Support Individual Decentralized Centralizing
Training Individual Intermittent Scheduled
Centralize d
Institutional
Custom Continuous
Copyright © 2004 Blackboard Inc.

?
University Teaching Development Centre
Victoria University of Wellington http://www.vuw.ac.nz/utdc/
14

Process category Brief description
Customer-Supplier Processes that directly impact the customer
Engineering
Project
Support
Organisation
Processes that specify, implement, or maintain a system and software product
Processes that establish the project, and coordinate and manage its resources
Processes that enable and support the performance of the other processes on the project
Processes that establish the business goals of the oganisation and develop process, product and resource assets which will help the organisation achieve its business goals
University Teaching Development Centre
Victoria University of Wellington http://www.vuw.ac.nz/utdc/
15

Process category Brief description
Learning
Development
Coordination
Evaluation
Organisation
Processes that directly impact on pedagogical aspects of e-Learning
Processes surrounding the creation and maintenance of e-Learning resources
Processes surrounding the oversight and management of e-Learning
Processes surrounding the evaluation and quality control of e-Learning through its entire lifecycle
Processes associated with institutional planning and management
University Teaching Development Centre
Victoria University of Wellington http://www.vuw.ac.nz/utdc/
16

1. Initial : The development process is characterized as ad hoc, and occasionally even chaotic. Few processes are defined, and success depends mainly on individual effort and heroics.
2. Repeatable : Basic project management processes are established to track cost, schedule, and functionality. The necessary process discipline is in place to repeat earlier successes on projects with similar applications.
3. Defined: Management and development activities are documented, standardized, and integrated into a family of standard processes for the organization.
4. Managed: Detailed measures of the process and product quality are collected so that the process and product are understood and controlled.
5. Optimizing: Continuous process improvement is facilitated by feedback from the process and from piloting innovative ideas and technologies.
Paulk, et al., 1993; Paulk, 1996
University Teaching Development Centre
Victoria University of Wellington http://www.vuw.ac.nz/utdc/
17

Level 0: Not done at all
Level 1: Ad-hoc processes
Level 2: Clear objectives for e-learning
Level 3: Defined process for development
Level 4: Ensuring the quality of both the e-learning resources and student learning outcomes
Level 5: Continual improvement
University Teaching Development Centre
Victoria University of Wellington http://www.vuw.ac.nz/utdc/
Marshall & Mitchell (2002)
18

Maturity Levels
Indicate
Process Capability
Contain
Key Process Areas
Achieve
Goals
Organised by
Address
Implementation or
Institutionalisation
Common Features
Contain
Key Practices
Infrastructure or
Activities
University Teaching Development Centre
Victoria University of Wellington http://www.vuw.ac.nz/utdc/
Describe
19

• Are there common practices that lead to effective adoption and support of e-learning?
• For example:
Institutional vision and e-learning strategy?
Standards based technical environment?
Centralised/decentralised support?
Catalogue of successful solutions?
Formal project standards?
Formal evaluation of resources
…
• How are these affected by the institutional context?
• What aspects of that context drive e-Learning success and failure?
University Teaching Development Centre
Victoria University of Wellington http://www.vuw.ac.nz/utdc/
20

:
• Encourages student-faculty contact
• Encourages cooperation among students
• Encourages active learning
• Gives prompt feedback
• Emphasizes time on task
• Communicates high expectations
• Respects diverse talents and ways of learning
Chickering and Ehrmann, 1996
University Teaching Development Centre
Victoria University of Wellington http://www.vuw.ac.nz/utdc/
21

• Institutional Support Benchmarks
• Course Development Benchmarks
• Course Structure Benchmarks
• Student Support Benchmarks
• Faculty Support Benchmarks
• Evaluation and Assessment Benchmarks
Institute for Higher Education Policy, 2000
University Teaching Development Centre
Victoria University of Wellington http://www.vuw.ac.nz/utdc/
22

Course Development Benchmarks
• Guidelines regarding minimum standards are used for course development, design, and delivery, while learning outcomes —not the availability of existing technology —determine the technology being used to deliver course content.
• Instructional materials are reviewed periodically to ensure they meet program standards.
• Courses are designed to require students to engage themselves in analysis, synthesis, and evaluation as part of their course and program requirements.
University Teaching Development Centre
Victoria University of Wellington http://www.vuw.ac.nz/utdc/
23

Development
Processes surrounding the creation and maintenance of e-learning resources
D1.CDB1a
Guidelines regarding minimum standards are used for course development, design and delivery
D2.ISB2
The reliability of the technology delivery system is as failsafe as possible
D3.CDB1b
Learning outcomes, not the availability of existing technology, determine the technology being used to deliver course content
D4.FSB1a
Technical assistance in course development is available to faculty
D5.FSB1b
Faculty are encouraged to use technical assistance when
(re)developing courses
D6.FSB2a
Faculty members are assisted in the transition from classroom teaching to online instruction
University Teaching Development Centre
Victoria University of Wellington http://www.vuw.ac.nz/utdc/
24

Level Initial 1 Planned 2 Defined 3
Guidelines regarding minimum standards are used for course development, design and delivery
The reliability of the technology delivery system is as failsafe as possible
Learning outcomes, not the availability of existing technology, determine the technology being used to deliver course content
Technical assistance in course development is available to faculty
Faculty are encouraged to use technical assistance when (re)developing courses
Faculty members are assisted in the transition from classroom teaching to online instruction
Managed
4
Optimising
5
University Teaching Development Centre
Victoria University of Wellington http://www.vuw.ac.nz/utdc/
Fully Adequate
Largely Adequate
Partially Adequate
Not Adequate
25

• Are the “Quality on the Line” benchmarks sufficient?
• How do we know they are valid?
• What else can we learn from the literature about high quality elearning?
• Preliminary scan indicates that there may be over one hundred practices...
• Potentially up to a dozen process categories
University Teaching Development Centre
Victoria University of Wellington http://www.vuw.ac.nz/utdc/
26

• Are the practices that were chosen, the characteristics that tend to make a process more capable?
• Do the practices cover all of the relevant capabilities defined at each level?
•
• Are the practices defined so as to be independent of each other?
Is this set sufficient to characterise a range of capabilities?
Do the practices represent the “universal truths” of process capability?
•
• Are the practices genuinely applicable to any process?
University Teaching Development Centre
Victoria University of Wellington http://www.vuw.ac.nz/utdc/
27

• Roadmap for institutions
• Mechanism for stimulating institutional engagement
• Support for planning
• Organising existing heuristics
• Basis for ongoing discussion
• Sector wide capability analysis...
University Teaching Development Centre
Victoria University of Wellington http://www.vuw.ac.nz/utdc/
28

University Teaching Development Centre
Victoria University of Wellington http://www.vuw.ac.nz/utdc/
29

University Teaching Development Centre
Victoria University of Wellington http://www.vuw.ac.nz/utdc/
30

University Teaching Development Centre
Victoria University of Wellington http://www.vuw.ac.nz/utdc/
31

“All models are wrong; some models are useful” stephen.marshall@vuw.ac.nz
geoff.mitchell@griffith.edu.au
The authors wish to acknowledge the kind support of the New Zealand
Ministry of Education Tertiary E-Learning Research Fund for supporting this research and Blackboard for their Framework and support in attending this conference.
University Teaching Development Centre
Victoria University of Wellington http://www.vuw.ac.nz/utdc/
32