1
Inukshuk Project –
The Triple “A” Approach to Lecture Intervention
www.newmediaworkshops.com/tripleAlecture/
Development Plan – Phase 1
August 8th, 2005
Revised August 22nd, 2005
Dr. Jack Jones,
Director of Instructional Development and Technology Mediated Learning, Nipissing University
jackj@nipissingu.ca
AND
Susan Lister,
Inukshuk Project - Assistant Researcher/Developer for Nipissing University
susan.lister@newmediaworkshops.com
Document Change History
Revised By
Date
Description
Susan Lister
August 22
- Added website portal address
- updated info in Resource section
- updated Partner information
2
Table of Contents
Inukshuk Project – The Triple “A” Approach to Lecture Intervention ...................................................... 3
Project Description...................................................................................................................................... 3
Background ................................................................................................................................................. 3
Image 1 – The Teaching Process Pathway ............................................................................................. 3
Explanation ............................................................................................................................................. 3
Teaching Strategies and Learning Activities .......................................................................................... 4
Scope Statement .......................................................................................................................................... 4
Our Definition of a Lecture..................................................................................................................... 4
Target Audience ...................................................................................................................................... 5
Instructional Challenge ........................................................................................................................... 5
Specific Goals and Objectives ................................................................................................................ 6
Proposed Learning Scenario ................................................................................................................... 6
Current Tool ............................................................................................................................................ 7
Usability Testing and Learning Evaluation ................................................................................................ 7
Proposed Project Partners and their Roles .................................................................................................. 8
Resource Requirements ............................................................................................................................ 10
A home for our Learning Object ........................................................................................................... 10
Training to improve PHP and MySQL skills ........................................................................................ 10
INUKSHUK Project Activity List and Projected Timeline ...................................................................... 11
Summary ................................................................................................................................................... 13
Appendix 1 – Classroom Checklist – Slightly Modified for Current L.O. ............................................... 14
Appendix 2 - CLOE Base Set of Questions: Student Questionnaire ........................................................ 19
Part 1: Background ............................................................................................................................... 19
a. Background Information ............................................................................................................... 19
b. Course Information ....................................................................................................................... 19
Part 2: Learning Value .......................................................................................................................... 20
Part 3: Value Added .............................................................................................................................. 20
Part 4: Usability of the Multimedia Learning Object ........................................................................... 21
Part 5: Technology Function ................................................................................................................. 21
Part 6: Open Questions ......................................................................................................................... 22
Interview Protocol: Faculty....................................................................................................................... 23
Interview Protocol: Instructional Designer and/or Developer .................................................................. 24
References ................................................................................................................................................. 25
3
Inukshuk Project – The Triple “A” Approach to Lecture Intervention
Project Description
Our learning object will address the professional development needs of instructors in post-secondary
settings. Recognizing that “lectures and delivery of content have a position of primacy in university
practice” (Phillips, 2005) and that lectures continue to be “relatively cost effective and useful in
providing a knowledge base” (Bligh, 1972 in Phillips, 2005), our Inukshuk Project will focus
specifically on helping instructors to present effective lectures.
Background
We are setting out on a journey to assist instructors in their quest to effectively support their students’
learning process. Our journey stems from the notion that students are the binding force to a teaching
framework that includes the following factors:
1. Goals - What are the knowledge, skills or curriculum objectives particular to your course?
2. Content - What information and knowledge is needed by the student to attain the outcome?
3. Assessment - What methods are used to know that the student has met the learning objective?
4. Resources - What classroom requirements, personal items, etc. are needed to relay the
information?
5. Teaching Strategy and Learning Activities - How is the information to be conveyed to the
student? How do the students interact with the information?
The following diagram can best capture our thoughts around the pathway an instructor follows when
preparing a unit for learning:
Explanation
Image 1 – The Teaching Process Pathway
An instructor starts with the learning goals
or objectives the student must attain. Next, the
instructor decides on the content necessary to
reach the goal. Now the instructor must
consider how the information will be assessed
and the resources available since both of these
have bearing on the teaching strategy chosen to
convey the information. The content,
assessment, and teaching strategy also affect
the learning activities possible to engage the
student with the process.
Once the instructor has chosen the teaching
strategy and learning activities, s/herepeats the
rotation of content to be presented, the
assessment, resources, and a review of the
goals – but now, in relation to the specific
Jones, 2005 teaching strategy and learning activities
selected.
4
Teaching Strategies and Learning Activities
Our product begins at phase 5 shown above, the Teaching Strategies and Learning Activities. We
wanted to create a support for a variety of instructors and realized that it is at phase 5 where we can have
the most impact across disciplines. Although there are basically three types of teaching strategies and
each strategy can include a variety of learning activities in which the student interacts with the curricular
content in some way (as shown in the table below), our product will address the Lecture strategy
specifically.
Table 1 – Teaching Strategies and Corresponding Learning Activities.
TEACHING STRATEGY
Instructor Led Lecture
POSSIBLE LEARNING ACTIVITIES
Watching and listening
Answering & Asking questions
Taking notes
Same as above
Demonstration
Student Led
Correspondence Course
Lab
Group Activity
Combined
Online Course
Debate
Discussion
Research, answer questions, write papers, etc.
Perform experiment, make observations, write up
experiment.
Brainstorm ideas, SWOT analysis of ideas,
Present final work
Moderate discussion, complete exercises,
read/post messages, participate in chat
Research, prepare statements, present rebuttals
Answer questions, listen to other student’s
responses, reply to other student’s answers.
Scope Statement
We plan to produce a learning object that will assist instructors to present effective lectures. Our
learning object will be based on the teaching framework shown above and will apply the following
“Triple A” intervention whereby an instructor can:
1. Become AWARE of lecture techniques that are consistent with what we know about how
a student learns.
And/Or
2. Analyse and compare their current lecturing methods to best practices and learning styles.
And/Or
3. Apply and Act on new lecturing techniques.
Our Definition of a Lecture
For the purpose of this project, a lecture is defined as follows:
At least twenty minutes of content delivery consisting mostly of one-way vocal transfer.
This means that:
1. An instructor may employ various teaching strategies within one class period (and in fact,
depending on the learning goals and content to be covered, we may encourage this).
2. Our learning object can be used to investigate/improve any twenty minutes of one-way
vocal transfer of content.
5
Target Audience
Our learning object will focus on university instructors since they often deliver content using the lecture
method. However, since we are not focusing on the delivery of specific content, any instructor
(secondary/post-secondary) who wishes to investigate the lecture teaching strategy and connected
learning activities could benefit. We see two reasons why an instructor may be interested in our learning
object:
1. S/he has an intrinsic desire to become the best they can at lecturing.
2. S/he has received less than acceptable student ratings and therefore is interested in
improving her/his lecturing technique (extrinsic desire).
The user characteristics that are pertinent to our learning object include:

COMPUTER LITERACY – Our instructors would be very familiar with searching on the
internet, navigating web pages and using email. They would not necessarily know how
to install a plug-in.

TIME AVAILABILITY – An instructor usually has prep-time built into their teaching
assignments so we will assume that an instructor could have at least a 30-minute window
to interact with the content within the learning object.

ACADEMIC ORIENTATION – We are going to assume that any of the information
within the learning object should be supported by research with links to the research
provided. Also, since most instructors are consumers of research, we will assume that
our instructors prefer reading text but in a printed form. They do not expect content to be
presented to them in a multimedia mode to capture or maintain their attention.

ASSESSMENT SCEPTICISM – Although some instructors may have an intrinsic
interest in improving their teaching skills, there is an equal fear that self-assessment may
be used against them or get into the hands of the wrong person.

TEACHING SKILL – Typically, instructors are not hired for their teaching skills;
however to achieve tenure they must receive positive student evaluations that are based
on teaching. Normally, instructors do not get formal training in teaching.
Instructional Challenge
There are basically two challenges we must address with our learning object. One is based on the
struggle between teaching strategies and how people learn while the second focuses on the requirements
of our user audience, instructors. Perhaps Phillips can best sum up the teaching/learning challenge - he
asks “In what ways is it possible to achieve a deep learning, student-centred approach to learning within
a traditional lecture (Phillips, 2005)”. The second focus is captured by this question: “How do we create
a learning object that the instructor culture will buy into and ultimately use?”
6
Specific Goals and Objectives
There are three main goals for this learning object based on the “Triple A” approach:
AWARENESS
 The learning object will show how the learning process and the lecture method can be
linked through research and knowledge of best practices.
ANALYSE
 Provide a safe, secure and meaningful opportunity to analyse current lecture methodology
being used by the instructor.
ACT

Provide direction for instructors to improve lectures.

Assist the instructor to prepare an action plan for growth.
Proposed Learning Scenario
The learning object’s “Triple A” framework allows an instructor to choose one of three options to
explore: AWARENESS, ANALYSIS or ACTION. Although an instructor may wish to start with the
“Awareness” option and then work through the Analysis and Action tracks, we have conceived of each
option as a separate entity to allow full flexibility to meet an instructor’s time constraints and overall
needs. For this reason, there will definitely be some overlap between the content of each option.
Table 2 – Triple A Framework Description
Triple A
Framework
AWARENESS
Description
Content
This stage is for the instructor who wants to become
more knowledgeable about how the lecturing method fits
into what we know about teaching and learning. The
awareness stage will include quick access to information
for the instructor who asks, “What does the research say
about lecturing and its role in learning” and “what are
some quick tips for improvement?”
 What we know about how people learn
 Resources specific to lecturing
 Research supporting our self-appraisal tool and
other content on the website.
ANALYSIS
This stage includes self-appraisal tools for the instructor
who wants to know how her/his current practice
compares with best practice. For this stage, the instructor
would be asking, “I wonder how my skills compare to
exemplary teachers and where do I need to improve?”
Self-Appraisal Tools
1) Lecture Preparation - An instructor will
compare how s/he prepares for a lecture with
how an expert lecturer approaches the task.
2) Lecture Delivery – An instructor will assess
their lecture delivery.
3) Reflection – An instructor will review their
personal beliefs about teaching, learning, and
how lecturing fits into these beliefs.
ACTION
This stage is for the instructor who wants to go beyond
analyzing their lecturing techniques. There are tools to
aid in the design and development of a good lecture and a
formalized development process is presented. This stage
answers the question, “What can I do to get better?”
 No/Go Tool for deciding upon lecturing as a
Teaching Strategy
 Lecture Development Tool for preparing,
implementing and evaluating your lectures.
 Formalized P.D. Process involving:
1) No/Go Tool
2) Lecture Development Tool
3) Self-Appraisal of Teaching/Learning Beliefs
(see above)
4) Self-Appraisal of Lecture Delivery (see
above)
5) Within 6 weeks start at #1 again.
7
Current Tool
Our learning object will be based, in part, on the “Classroom Observation Checklist” developed by Dr.
Jack Jones (2000). The classroom checklist has been used with instructors who have indicated a desire
to improve their university teaching. The checklist approaches the teaching process through five
domains (verbal, auditory, visual, cognitive and physical) that address student learning styles and the
learning environment. We will be altering the current Classroom Observation Checklist tool to
specifically address the lecture teaching strategy. To review the complete checklist, see Appendix 1. A
sample question from each of the domains is shown below:
Table 3 – Teaching Strategies and Corresponding Learning Activities
Verbal Domain
Speaks at a reasonable pace
1
2
Not at all
Somewhat
3
4
5
6
Moderately
Much
Very Much
N/A
Auditory Domain
Speaks at sound level appropriate to room
1
2
3
Not at all
Somewhat
Moderately
4
5
6
Much
Very Much
N/A
Visual Domain
Puts the outline of lecture/session on board or overhead screen prior to lesson beginning
1
2
3
4
5
6
Not at all
Somewhat
Moderately
Much
Very Much
Cognitive Domain
Reviews topics covered in previous class at beginning of the session
1
2
3
4
5
Not at all
Somewhat
Moderately
Much
Very Much
Physical Domain
Provides sufficient seating and desks for all students in the classroom
1
2
3
4
5
Not at all
Somewhat
Moderately
Much
Very Much
N/A
6
N/A
6
N/A
Usability Testing and Learning Evaluation
As with all the Inukshuk Learning Objects, our Lecture Intervention learning object will follow the
prescribed evaluation plan that will include both usability and learning evaluation. The Inukshuk
guidelines we have received provide the following explanation:
Learning Evaluation
The purpose of the learning evaluation is to understand how well the
learning objects are assisting learners in understanding the concepts that are
presented in the learning objects. The main goal of the learning evaluation
is to determine if the learning outcomes and objectives are being realized.
The following are the research questions that will aim to be answered in this
evaluation:
8
Do the learning objects provide a positive learning value to the students?
Do the learning objects provide an added value to the students?
Are the learning objects easy to use?
Are the learning objects prohibitive to use due to the technology?
Any other additional subjective information about the learning objects.
Usability Testing
The purpose of the usability testing is to examine issues such as ease-of-use,
navigation, satisfaction of use, and the overall perceptions of the learning
objects being tested. The usability testing is meant to reveal any
deficiencies with the interface or interaction challenges that the user may
face when working with the learning objects. This testing will illustrate
potential improvements that could be made to the system to enhance the
overall efficiency and effectiveness of the learning objects.
(Holbrook and Palmer, 2005)
The evaluation of our learning object will follow the Inukshuk Project requirements with a few slight
modifications since our object focuses on instructors while most of the Inukshuk learning objects are for
students. See Appendix 2 for the Inukshuk/CLOE Base Set of Evaluation Questions.
Proposed Project Partners and their Roles
We have decided that the best role for our partners would be in reviewing and evaluating the project as it
passes through various development phases. As an item is sent out to a partner, it will include a specific
list of the tasks that need to be completed for evaluation. For instance, in the first phase, the partners will
be asked to review a document. This first phase document will be sent with a list of points that need
review or investigation. In the second and third phases, the partners will review a prototype. The
evaluation task for the prototype phases will include usability testing as well as evaluating the learning
components. For all evaluation tasks requested of the partners, a testing/evaluation document will be
provided that clearly identifies the procedures and tasks required for completion.
Milestones which require the involvement of the Partners
Project Plan Evaluation
Evaluate Project Planning Phase 1 Document
Prototype Development – Phase 1 - Design
Test Low-fidelty Paper Prototype (flow-chart, storyboards,
etc.)
Test High-Fidelity Prototype (i.e. use powerpoint to show
interactivity)
Evaluate Design Doc. & Revisit Complete Planning Doc.
Prototype Development – Phase 2 - Production & Testing
Alpha Version Testing and Debugging
Beta Version Testing and Debugging
EVALUATION
Conduct Evaluation
Compile Results
Submit Results
Proposed Date when
Expected Date of Return
Partners will Receive
from Partners
Item
August 8
August 20
September 30
October 10
October 20
October 30
November 5
November 30
December 30
December 10
January 10
January 30
February 28
March 10
February 28
March 10
March 10
9
We will be inviting the following universities to become partners:
University
Reason for Involvement
Nanning, China
Involved in professional development of professors
with Nipissing University
University of Ottawa Developed Instructor Development Website that
contains excellent research findings and best-practice
videos from 3M Instructors.
http://www.facultydevelopment.ca/ info@iathe.org
Murdoch University,
Australia
Dr. Phillips wrote, “Challenging the Primacy of
Lectures” a paper that investigated the lecture mode
of teaching.
Contact Person
Dr. Christian Blanchette, Director,
Teaching and Learning Support
Service of the University of
Ottawa blanchette@uottawa.ca
Tel:(613) 562-5300
Adam Caron at caron@iathe.org
Dr. Rob Phillips
r.phillips@murdoch.edu.au
*Dr. Phillips has agreed to “limited
assistance” as of August 22, 2005
Memorial University,
Newfoundland,
Canada
Contact through Camp CLOE
Albert Johnson
Higher Colleges of
Technology, United
Arab Emirates
Since a learning object is to be ‘reusable’. Having
international partners will be advantageous.
Peter Hatherly Green,
Dubai Men’s College
peter.greene@hct.ac.ae
University of Central
Queensland,
Australia
Since a learning object is to be ‘reusable’. Having
international partners will be advantageous.
Yvonne Toft
y.toft@cqu.edu.au
albertj@mun.ca
*Albert was on holidays when the first
invitation went out.
Dr. Les Killion
l.killion@cqu.edu.au
Dr. Deborah Peach
d.peach@cqu.edu.au
University of
Manitoba,
Canada
Brock University,
The University of Manitoba is a recent member of
CLOE (Cooperative Learning Object Exchange):
http://cloe.on.ca/index.html - although not required,
we would prefer to have some CLOE members as our
Partners.
Develop ties for future research projects and Joint
Education PhD work.
Peter Tittenberger.
Learning Technologies Centre
University of Manitoba
tittenbe@cc.umanitoba.ca
Dr. Renee Kuchapski
Faculty of Education at Brock
University
Renee.Kuchapski@Brocku.ca
*As of Aug. 22, we have not heard back from
Dr. Kuchapski.
10
Resource Requirements
A home for our Learning Object
In preparation of this planning document, we have come to believe that our learning object will most
probably come to fruition more as a complete website tied to a database rather than a more ‘portabletype’ learning object that can be easily copied and placed on any number of websites or reproduced on a
CD-ROM. For this reason, one of our major tasks in Phase 1 – Design will be to find a host for our
learning object that will include easy access during the development phases to FTP, MySQL with at
least two free databases, PHP, and perhaps Moodle.
Peter Tittenberger, Instructional Technology Coordinator of Information Services and Technology
at the University of Manitoba has come forward and offered personnel who can help out by providing
server space.
Training to improve PHP and MySQL skills
As our learning object begins to take shape within this planning document, it is becoming clear that the
lead developer may require some additional training in PHP and MySQL to satisfy the learning
objectives we are setting out. Approximately 30 hours of a mentor’s time will be needed whereby
programming ideas will be presented and reviewed.
Peter Tittenberger, Instructional Technology Coordinator of Information Services and Technology
at the University of Manitoba has come forward and offered personnel who can help out in the PHP
mentoring aspect.
Distributed Meeting Possibilities
We are in the process of finalizing our list of partners for this project and as things have worked out we
will have a very qualified international team working on the learning object. As a result, we will need to
investigate some virtual meeting options such as “gotomeeting”, gnome meeting, webex, skype, etc. in
order to allow regular synchronous meetings to occur.
Assumptions- Constraints- Limitations and Issues

Although we feel that a major strength in our proposed product is the evaluation cycle that
includes regular review by our partners, we feel that this can also be a limitation. The partners
may not be timely in their reviews or they may not be “on the same wavelength” with the
product as we are. By including many partners to review and evaluate the various stages, we
hope to maximize the possibility of timely feedback while ensuring we are not held up at any
one stage.

Another issue is that the product we have proposed is a rather unique learning object. Our
product will be more of a complete package with parts of it that can be “reused” like a typical
learning object or parts of the package might be ‘linked to’ in order to support other faculty
development initiatives across the country and globally. In particular, the items that may be used
on their own could be the No/Go Lecture Tool, the Lecture Self-Appraisal Tools and the Lecture
Planning Tool.

The fact that our product must be developed and evaluated within the same school year, could
become an issue since it only leaves one month to evaluate a product that is meant to support a
faculty member over approximately a 3 month professional development cycle. This means that
in order to evaluate the product in a realistic setting we must not only find faculty members who
wish to improve their lecturing but they must be willing to start down an intervention path in
January, when our product is to be evaluated.
11
INUKSHUK Project Activity List and Projected Timeline
P–
Indicates Role for Partner
DESCRIPTION OF TASKS
PLANNING - Phase 1
Project Description
Preliminary Research on the Concepts (See list of URL’s and Documents –
Appendix 1)
Identify Target Audience (Develop Personas/Learner Profiles)
Description of Current Tool (See Appendix 2 for Actual Tool)
Identify Assumptions- Constraints- Limitations and Issues
Prepare Scope Statement
Instructional Challenge
Specific Goals and Objectives
Propose Evaluation Criteria (How will we know that we are successful?)
Propose Learning Scenario
Project Activity List and Projected Timeline
Provide Project Planning Phase 1 Document for Review
P
Evaluate Project Planning Phase 1 Document
Make changes to Document as noted in evaluations and identify committed
partners.
DATE TO BE
COMPLETED BY
August 20th
August 30th
12
PLANNING - Phase 2
Identify Partners & Confirm Involvement
Propose Partners -explain roles
Sign off Commitment to Participate
Research Existing Instructor Development Methods/Objects
Identify Specific Resource Requirements and Functional Specifications
Create Evaluation Plan (Usability and Learning Evaluation)
Identify Tasks and Milestones (Gantt Chart)
Prepare Budget
Provide Project Planning Phase 2 Document for review
P
Evaluate Project Planning Phase2 Document
Make changes to Document as noted in evaluations
Submit Required Forms to Ethics Committee
Revisit and Revise Phase 1 Documentation
Present Finalize Project Plan (includes Phase 1 & 2)
PROTOTYPE DEVELOPMENT – Phase 1 - Design
Instructional Design
Flowchart
Activity Development
Storyboard
Layout and Navigation
Information Design
Prepare Scripts for Screen Text
Create Paper Prototype
Prepare Low-fidelity Paper Prototype
P
Test and Revise Low-fidelty
P Evaluate Paper Prototype & Design Documents
Review Paper Prototype against CLOE Submission Guidelines
Revise Paper Prototype
Test High-Fidelity Prototype (i.e. use powerpoint etc. to show interactivity)
Evaluate Design Doc. & Revisit Complete Planning Doc
Prototype Development – Phase 2 - Production & Testing
Graphics, programming, Screen Text
Produce Testing Plan for Alpha and Beta Versions of Product
P
Send out for review by Partners
Alpha Version
P
Testing and Debugging
Make Revisions
Beta Version
P
Testing and Debugging
Make Revisions
Present Final Product
EVALUATION
P Conduct Evaluation
P Compile Results
P Submit Results
P – Indicates Role for Partner
September 10th
September 15
September 30th
October 20
November 5
December 30th
January 30th
February 28th
January 30th
February 28th
March 10th
13
Summary
The learning object we have proposed will explore the lecture teaching strategy. Our triple “A”
approach provides the instructor with three different interventions – the first being AWARENESS in
which an instructor, interested in improving his/her lecturing technique, will have access to information
about ‘best practice’ and how the lecturing method blends with what we know about how students learn.
In the second intervention, the instructor will have the opportunity to ANALYZE and assess his/her
current lecturing methods while the third option provides the instructor with many tools to ACT on
improving lectures including an action plan for growth.
As part of the Inukshuk project, our learning object project will follow the prescribed path identified in
the Inukshuk guidelines. In particular, we will seek out partners to assist in the evaluation and
assessment of our learning object and we will adhere to the Inukshuk deadline of March 2006.
14
Appendix 1 – Classroom Checklist – Slightly Modified for Current L.O.
Classroom Observation Checklist (Teaching Process Only)
1. Verbal Domain
a) Speaks at a reasonable pace
1
2
Not at all
Somewhat
b)
c)
d)
e)
f)
g)
h)
Speaks clearly
1
2
Not at all
Somewhat
3
Moderately
4
Much
5
Very Much
6
N/A
3
Moderately
4
Much
5
Very Much
6
N/A
Asks questions, pausing long enough for learners to reflect and then respond
1
2
3
4
5
Not at all
Somewhat
Moderately
Much
Very Much
6
N/A
Verbalizes everything that is visually presented
1
2
3
4
Not at all
Somewhat
Moderately
Much
5
Very Much
6
N/A
Emphasizes key points by pausing, speaking slowly, modulating voice
1
2
3
4
5
Not at all
Somewhat
Moderately
Much
Very Much
6
N/A
Explains subject matter in familiar, colloquial language
1
2
3
4
Not at all
Somewhat
Moderately
Much
5
Very Much
6
N/A
Speaks in dramatic or expressive way at appropriate times
1
2
3
4
Not at all
Somewhat
Moderately
Much
5
Very Much
6
N/A
Relates humorous anecdotes at appropriate times
1
2
3
4
Not at all
Somewhat
Moderately
Much
5
Very Much
6
N/A
***************************************************************************
2. Auditory Domain
a)
Sound level of AV equipment suitable to room
1
2
3
4
Not at all
Somewhat
Moderately
Much
5
Very Much
6
N/A
15
b)
c)
d)
e)
Speaks at sound level appropriate to room
1
2
3
Not at all
Somewhat
Moderately
4
Much
5
Very Much
6
N/A
Speaks to chalkboard as opposed to audience
1
2
3
Not at all
Somewhat
Moderately
4
Much
5
Very Much
6
N/A
Speaks to screen as opposed to audience
1
2
3
Not at all
Somewhat
Moderately
4
Much
5
Very Much
6
N/A
Speaks to overhead projector as opposed to audience
1
2
3
4
Not at all
Somewhat
Moderately
Much
5
Very Much
6
N/A
Ensures that all learners can hear each other’s questions or comments (getting learners or teacher in
specific situations to repeat them if necessary)
1
2
3
4
5
6
Not at all
Somewhat
Moderately
Much
Very Much
N/A
f)
3. Visual Domain
a)
Puts outline of lecture/session on board or overhead screen prior to lesson beginning
1
2
3
4
5
6
Not at all
Somewhat
Moderately
Much
Very Much
N/A
b)
Smiles and/or appears to be look relaxed
1
2
3
Not at all
Somewhat
Moderately
4
Much
5
Very Much
6
N/A
c)
Technical terms, all new concepts and potentially difficult words are presented visually
1
2
3
4
5
6
Not at all
Somewhat
Moderately
Much
Very Much
N/A
d)
Overheads presented in colours and fonts that can be easily read
1
2
3
4
5
Not at all
Somewhat
Moderately
Much
Very Much
6
N/A
16
e)
f)
g)
h)
Uses gestures to demonstrate emphasis
1
2
3
Not at all
Somewhat
Moderately
5
Very Much
6
N/A
Uses graphs or diagrams to facilitate explanation at appropriate times
1
2
3
4
5
Not at all
Somewhat
Moderately
Much
Very Much
6
N/A
A/V material readily seen by all learners
1
2
3
Not at all
Somewhat
Moderately
4
Much
5
Very Much
6
N/A
Videos are captioned
1
2
Not at all
Somewhat
4
Much
5
Very Much
6
N/A
Reviews topics covered in previous class at beginning of the session
1
2
3
4
5
Not at all
Somewhat
Moderately
Much
Very Much
6
N/A
Uses advance organizers (identical to component 3a)
1
2
3
4
Not at all
Somewhat
Moderately
Much
5
Very Much
6
N/A
Shows respect for all students
1
2
3
Not at all
Somewhat
Moderately
5
Very Much
6
N/A
3
Moderately
4
Much
4. Cognitive Domain
a)
b)
c)
4
Much
d)
Is aware of all students in all locations, not just those who take an active role in the lecture and/or
activities
1
2
3
4
5
6
Not at all
Somewhat
Moderately
Much
Very Much
N/A
e)
Uses questions to encourage learning
1
2
3
Not at all
Somewhat
Moderately
4
Much
5
Very Much
6
N/A
17
f)
g)
h)
i)
j)
Asks for input from a variety of learners
1
2
3
Not at all
Somewhat
Moderately
4
Much
5
Very Much
6
N/A
Encourages learners to ask questions and express opinions during class
1
2
3
4
5
Not at all
Somewhat
Moderately
Much
Very Much
6
N/A
Responds to learners questions specifically
1
2
3
Not at all
Somewhat
Moderately
4
Much
5
Very Much
6
N/A
Incorporates learners ideas, comments into lecture
1
2
3
4
Not at all
Somewhat
Moderately
Much
5
Very Much
6
N/A
Defines new or unfamiliar terms
1
2
3
Not at all
Somewhat
Moderately
5
Very Much
6
N/A
4
Much
k)
Uses concrete/ relevant examples or analogies to explain concepts and principles
1
2
3
4
5
6
Not at all
Somewhat
Moderately
Much
Very Much
N/A
l)
Uses visual aids (board /overhead, illustrative materials, technology) to explain new or unfamiliar
terms
1
2
3
4
5
6
Not at all
Somewhat
Moderately
Much
Very Much
N/A
m)
Repeats new/potentially difficult concepts more than once during session
1
2
3
4
5
Not at all
Somewhat
Moderately
Much
Very Much
6
N/A
Checks for understanding before proceeding to next topic
1
2
3
4
Not at all
Somewhat
Moderately
Much
6
N/A
n)
o)
5
Very Much
Clearly indicates transition from one topic to the next within a single lesson
1
2
3
4
5
Not at all
Somewhat
Moderately
Much
Very Much
6
N/A
18
p)
The selection process used to organize students into various activities (i.e. group work,
presentations) are done in a fair and consistent manner
1
2
3
4
5
6
Not at all
Somewhat
Moderately
Much
Very Much
N/A
q)
Implements group activities at appropriate times
1
2
3
4
Not at all
Somewhat
Moderately
Much
r)
s)
5
Very Much
6
N/A
Summarizes topics “covered” in current class prior to closure of session
1
2
3
4
5
Not at all
Somewhat
Moderately
Much
Very Much
6
N/A
Indicates out of class resources relevant to lecture or activities (WEBCT, Blackboard, other CMS,
website URL’s, books)
1
2
3
4
5
6
Not at all
Somewhat
Moderately
Much
Very Much
N/A
5. Physical Domain
a)
b)
c)
Basic comforts (lighting, temperature) present in the room
1
2
3
4
Not at all
Somewhat
Moderately
Much
5
Very Much
6
N/A
Sufficient seating and desks for all students in the classroom
1
2
3
4
5
Not at all
Somewhat
Moderately
Much
Very Much
6
N/A
Lab and computer equipment is available for learner use at appropriate times
1
2
3
4
5
Not at all
Somewhat
Moderately
Much
Very Much
6
N/A
Miscellaneous
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
JJ Revised March 2005
19
Appendix 2 - CLOE Base Set of Questions: Student Questionnaire
We would like to gather feedback about your experiences with the [multimedia learning object]
specifically concerning its value for you as an individual learner and for any added value the
[learning object] may bring to this course. We also seek your feedback on how usable you found
the tool to be and how well the technology functioned during its use. The information you provide
will be used to improve or enhance the [multimedia learning object] where needed for the benefit
of future students taking this course.
We appreciate your willingness to participate in this questionnaire.
Part 1: Background
a. Background Information
Please tell use a little bit about yourself by checking the appropriate selection:
Gender:
Female: ______
Age:
□
Male: ______
□
Under 20
21 to 25
□
26 – 30
Undergraduate Term:
1 A/B: ______
4 A/B: ______
2 A/B: ______
5 A/B: ______
Graduate Program:
Masters: ______
PhD: ______
□
30+
3 A/B ______
Other (diploma, certificate, etc.) – please specify: _______________________
b. Course Information
What is the format of your class?
□
□
□
Distance Education, On-line
Traditional, On-campus
Other / Combined
20
Part 2: Learning Value
Please select the number that best describes the learning value of the [multimedia learning object].
Please answer all the questions.
1
strongly disagree
2
disagree
3
undecided
4
agree
5
strongly agree
N/A
The incorporation of the [multimedia learning object] into the course…
1. helped me learn the new skills and concepts for this course.
1
2
3
4
5
N/A
4
5
N/A
2. helped me learn the materials in a new way.
1
2
3
3. helped in that it reduced the amount of time spent studying text or notes in order to grasp the
material to be learned.
1
2
3
4
5
N/A
4
5
N/A
4
5
N/A
4. helped me learn these concepts at my own pace.
1
2
3
5. helped me to visualize the concept better.
1
2
3
6. I found that the multimedia learning object was well integrated with the other course
components offered for this topic area (i.e., lecture, textbook, etc.).
1
2
3
4
5
N/A
Part 3: Value Added
Please select the number that best describes the value added of the [multimedia learning object].
Please answer all the questions.
1
strongly disagree
2
disagree
3
undecided
4
agree
5
strongly agree
N/A
When using the [multimedia learning object]…
7. I gained as much information from this [multimedia learning object] as I would from a lecture
explanation of the same concept.
1
2
3
4
5
N/A
8. helped me to better understand how to [activity in the multimedia learning object].
1
2
3
4
5
N/A
9. I was able to work through examples in a way that would not have been possible by attending a
lecture or reading a textbook.
1
2
3
4
5
N/A
10. I was able to [solve extra problems, experience situations, etc.] that I otherwise wouldn’t have
done in the classroom or studying from a textbook.
1
2
3
4
5
N/A
21
Part 4: Usability of the Multimedia Learning Object
Please select the number that best describes the usability of the [multimedia learning object].
Please answer all the questions.
1
strongly disagree
2
disagree
3
undecided
4
agree
5
strongly agree
N/A
4
5
N/A
I found that…
11. the [multimedia learning object] was easy to use.
1
2
3
12. the navigation within the [multimedia learning object] was clear and easy to use.
1
2
3
4
5
N/A
13. I was able to fully use the [multimedia learning object] by following the instructions provided.
1
2
3
4
5
N/A
14. the material presented in the [multimedia learning object] flowed in a logical order.
1
2
3
4
5
N/A
Part 5: Technology Function
Please select the number that best describes the technical functioning of the [multimedia learning
object]. Please answer all the questions.
1
strongly disagree
2
disagree
3
undecided
4
agree
5
strongly agree
N/A
When using the multimedia learning object, I found….
15. I was not disadvantaged because I possess adequate computer skills.
1
2
3
4
5
N/A
16. I did not miss important information because the technology worked correctly.
1
2
3
4
5
N/A
5
N/A
4
5
N/A
4
5
N/A
17. the hardware/software requirements did not present a problem for me.
1
2
3
4
18. I have not used this software tool due to technical difficulties.
1
2
3
19. I did not use this software tool for other reasons.
1
2
3
22
Part 6: Open Questions
Please think of the [multimedia learning object] when answering the questions below.
20. What were the main strengths of the [multimedia learning object]?
21. What were the main weaknesses of the [multimedia learning object]?
22. In your estimation, is it desirable to use technology to support teaching and learning in
campus-based courses? Why or why not?
23
Interview Protocol: Faculty
Name of Institution:
_________________________________________
Name of Faculty Member:
_________________________________________
Name of Learning Object:
_________________________________________
Name of Interviewer:
_________________________________________
1. Is this a learning object that you developed or helped to develop? How were you involved
in its development?
2. What do you feel is the greatest reward for creating/using this learning object?
3. Have you used this learning object, or other like it, before in previous classes or other
courses? If yes, what classes or courses?
4. Can you tell me how you use this learning object/tool in your teaching?
5. If its use is optional for students, do you know how many of them used it? If not, can you
speculate on this?
6. Do you have a sense of how helpful it was for them?
7. What kinds of comments/feedback did students make about the learning object, if any?
8. If you were going to make changes to this learning object or to the way you have students
use it, what would these changes be?
9. Can you think of other courses in which this learning object could be used? If so, how
would it have to be changed for repurposing it, if at all?
10. Is there any other information that you would like to gather, such as comparisons of student
performance with and without the use of the learning object?
11. Is there anything else that we did not touch on that stands out for you about your
experience with this learning object?
24
Interview Protocol: Instructional Designer and/or Developer
Name of Institution:
_______________________________________
Name of Instructional Designer:
_______________________________________
Name of Learning Object:
_______________________________________
Name of Interviewer:
_______________________________________
1. How were you involved in the development of this learning object? What was your role and
responsibilities?
2. What design and development process did you utilize when creating the learning object?
3. What testing and evaluation approach did you utilize for validating the effectiveness of the
learning object?
4. What did you learn from the testing and evaluation?
5. What do you think was the greatest success in creating this learning object?
6. What were the greatest challenges in creating this learning object?
7. How would you describe the instructional design that is a part of this learning object?
8. Do you think that this learning object could be reused in other courses? How was the issue
of reuse considered when the learning object was created?
9. What advice would you give someone who was creating a learning object for the first time?
10. Is there anything else that we did not touch on that stands out for you about your
experience with this learning object?
25
References
Gold, J.R., Jenkins, A., Lee R., Monk, J. Riley, J., Shepherd, I., Unwin, D. “Teaching Geography In
Higher Education, A Manual of Good Practice - Retrieved August , 2005
http://www2.glos.ac.uk/gdn/gold/ch2.htm
Phillips, R. A. (2005). Challenging the Primacy of Lectures: the Dissonance between Theory and
Practice in University Teaching. Journal of University Teaching and Learning Practice, Volume 2,
Issue 1, 2005 - Retrieved June 15, 2005
jutlp.uow.edu.au/2005_v02_i01/phillips003.html
Holbrook, J., Palmer A., “Evaluation Plan - Biology – Learning Objects for Introductory Biology”
unpublished paper. June, 2005. Retrieved from Inukshuk Project Work Site – July, 2005.