Brant Knutzen
Learning Designer
Faculty of Education
University of Hong Kong

Social Constructivism

Transactivity

Building engagement

Assessment methods

Participation rating

Measuring engagement

CMC : Computer-mediated Conferencing
 Synchronous : “chat”, or instant messaging
 Asynchronous : “discussion forum”

Asynchronous
 Participants take turns posting
 Minutes, hours, or days apart
 More time to digest, research, compose response

The effective construction of knowledge is a
product of a functional collaborative group
producing artifacts for public display and use

Efficacy has been found to be linked to the
process that learners utilize in working on the
task together (Fischer et al 2002)

Process: social negotiation of arguments
and argument sequences (Leitão 2000; Voss & Dyke 2001)

John Biggs captured the educational value of
discussion when he stated:
"Good dialogue elicits those activities that
shape, elaborate, and deepen understanding“
(Biggs 1999 p. 5)

Transactivity: the method by which students
build on the contributions of their fellow
learners
(Berkowitz & Gibbs 1983)

Transactive communication:
 Participants respond to and build on each other’s
contributions
 Peer exchange of information and ideas
 Social negotiation of knowledge
 Each participant brings their own experiences to
apply to a common educational goal

A key theoretical construct for measuring collaboration

How can we formulate the instructional design
conditions which consistently result in more productive
and transactive learning activities?

How can we describe it in easily grasped ways?
 Quantitative
 Qualitative

An online discussion is a great formative
learning activity for building depth of student
understanding, or supporting project work

Highest potential for social construction of
knowledge and transactive collaboration

Unfortunately, this activity is also the most
likely to FAIL! Why?

Case study of one course (Knutzen, 2007)
▪
▪
▪
▪
International school in Hong Kong – secondary level
1-to-1 laptop blended learning environment
Introduction to Psychology course
Sample size = 24

Investigation of instructional design conditions to
achieve a highly productive online discussion

At start of study, average student production in
optional online discussions = 1 post

Four conditions to achieve productive online
discussions:
1. Teacher facilitated social formation of small groups
2. Class time to initiate oral and online discussion interaction
3. Setting open-ended, challenging topic questions that
encourage discussion and debate
4. Assessment system that reinforces production and peer
interaction

At end of study, average student
production: over 10 posts per discussion!

Over the past six years:
 Extensive use of the online discussion design
 Full-time instruction of secondary students
▪ 1-to-1 laptop environment
▪ IT classes
▪ Psychology
 Implemented at University of Hong Kong: 2009
▪ MSc IT in Education course on eLearning
 Implemented in 12 courses at Lingnan Univ : 2010
▪ Business, History, Social Studies, Philosophy, English

Design continues to result in good production


One to three questions around one topic or area
of content / concepts
Advantage of multiple questions:
 Instructor can design a “gradient” of difficulty which
can elicit a range of student answers
▪ From basic knowledge -> higher-order thinking skills (HOTS)
▪ Use a taxonomy of active verbs to specify the levels of
understanding expected in answers (Blooms Revised, SOLO)
▪ Examples: Multi-structural (list, describe, classify)
Relational (compare/contrast, explain, analyze, relate)
Extended Abstract (hypothesize, generate, reflect)

Objective of multiple questions:
 Make discussion accessible to all students
 Challenge the advanced students

Other topic question gradients found to be useful:
 Concrete facts -> abstract concepts
 Textbook context -> personal context (unique answers!)
Example of a Topic Question gradient:
Can you demonstrate what you have learned in your study of
the Porter management models?
1.
2.
3.
Can you list and describe the Porter models? (Basic understanding )
.
How can you compare Porter's models? (Relational understanding)
Can you relate these models to each other in several ways, or on several
dimensions?
.
Based on these models, can you create your own model?
What factors do you theorize are important, and why?
(Extended abstract understanding)

Traditional – teacher-assessed subjective marking
 Review contributions by each student
 Award mark based on:
▪ Participation – any contribution to discussion
▪ Interaction - responding and seeking feedback
▪ Transaction – sharing / exchanging useful information and resources
▪ Transformation – perspective change due to interaction with others

Best method for summative assessment

Should a discussion be a summative activity?

Criteria examples:
▪ Locate, organize, analyze, evaluate, synthesize, and
ethically use information from a variety of sources and
media
▪ Interact, collaborate, and publish with peers, experts, or
others employing a variety of digital environments and
media
▪ Develop cultural understanding and global awareness by
engaging with learners of other cultures
▪ Contribute to project teams to produce original works or
solve problems
Source: ISTE NETS for Students

A highly productive discussion can easily
produce over 200 posts a week!

A teacher can become a victim of their own
success
 How much time can they devote to quantitative
marking?
 How much time remains for qualitative
feedback?

Desired graduate attributes:
 Critical thinking skills
 Excellent cooperative skills
▪ Integrity
▪ Personal responsibility

Subjective peer-assessment can directly
address the development of these attributes
 Requires student training
 Requires review and evaluation by teacher

Possible problems:
 Revenge grading : 報復
▪ “you gave me a low grade, I will give you a low grade”
 Back-scratching : 賄賂
▪ “If you give me a high grade, I will give you a high grade “

Teacher evaluation of forums: time intensive

Peer evaluation of forums: requires monitoring

Subjective evaluation of formative work tends
to distort student learning

Optional forums do not get sufficient
participation to create viable discussions

My solution: automate the rating based on
participation

No subjective judgment, just rating using a systematic method:

Moodle can automatically average these grades!

Moodle averages the ratings
 Grades produced by participation:
▪
▪
▪
▪
▪
▪
▪
▪

One post = 6
-> DTwo posts = 8
-> BThree posts = 8.6 -> B
Four posts = 9
-> AFive posts = 9.2
-> ASix posts = 9.33
-> A
Seven posts = 9.42 -> A
Eight posts = 9.5 etc
More Q&A participation = higher grade
Number of posts -> Forum Grade
10
9
F
o
r
u
m
8
7
6
5
G
r
a
d
e
4
3
2
1
0
0
1
2
3
4
5
6
7
8
9
10
11
12
13
Number of discussion posts
14
15
16
17
18
19
20
21

This assessment method for assessing a
forum creates a foundation of participation:
on average over 90% of students will engage

As always, the teacher’s role is to create the
educational value of the activity:
 Set up challenging topic questions to guide
exploration
 Maintain a “visible presence” in the forum activity
 Successfully moderate the discussion

Salmon’s 5-stage
model of
eModeration

Teacher guides
students up
the levels
toward
knowledge
development
Source: http://www.atimod.com/e-moderating/5stage.shtml

The Moodle LMS handles the awarding of
quantitative marks for participation

Teacher can focus on quality:
 Monitor progress
 Guide the discussion
 Challenge students
 Time for thoughtful qualitative
feedback
▪ Minimum: 30 mins to 1 hour each week per discussion

Motivation on three levels:
1. Constructive alignment between formative
discussions and summative final assignments
Examples:
▪ Reflective posts build towards a Learning Portfolio blog
▪ Group collaboration on final project development
2. Social Constructivism: a sense of social obligation
to support the group
3. Quantified Participation: each contribution is
counted as participation towards a forum grade

Automates the awarding of participation points
 Reduces technical requirements (modifying roles)
 No need to train the students
 No need to monitor rating accuracy
 No need to motivate student ratings of posts
 Finished development and testing in Dec 2011
 Currently in trials at HKU

Planned release as open-source project soon

Quantitative:
▪
▪
▪
▪

Production = Total number of discussion posts / n
Interactivity = Total number of feedback posts / n
Group Activity = Total number of discussion posts / # topics
Transactivity = Production × Interactivity
Qualitative: a new type of graphical
representation – the “Participation Map”
 Developed at Lingnan University in 2011
 Continued development as open source project

New Moodle plug-in automates the production of
the map of discussion activity:
 Quantitative statistics
 Qualitative graphic display: a “data portrait”

Useful for displaying discussion activity
 Feedback to students in the class (who is working?)
 Feedback to teacher (what approach is working?)
 Anonymous mode for reporting research results
Time scale from Day 1 to Day 14
Overall discussion statistics
Red line shows rating cutoff
Students can interact in
multiple groups
Group comparison stats
Student profile pictures

Quantitative ratings of overall discussion

After install on Moodle server, this plugin is
only available to those with Teacher roles

New option on Course
Administration block

Select the
forum, and
then plot type:
 Normal plot
for feedback
to teacher and
students
 Anonymous
for reporting
results outside

Currently setting up a website to make these
open source projects available to the Moodle
educational community: Brant.Knutzen.se

Goal is to release both as free open source
plugins for Moodle 2 by Sep 2012

Q & A ??

My email is brantknutzen@gmail.com