Part I: The 3 T’s of Online Assessment:
Tools, Techniques, and (Saving) Time
Curtis J. Bonk, Professor, Indiana University
President, CourseShare http://php.indiana.edu/~cjbonk; cjbonk@indiana.edu
Vanessa Paz Dennen, Assistant Professor
Florida State University http://www.vanessadennen.com
, vdennen@fsu.edu

 Detail online assessment techniques
 Discuss how to match learning activities with learner assessments
 Examine instructor time and comfort issues
 Discuss ways to limit and detect cheating and plagiarism
 Document online tools and resources for assessment

(with some time-saving tips added in…)

 You take an online class.
 You read some Web pages.
 Maybe you watch some videos or hear some audio clips.
 Maybe you ponder some study review questions.
 You take a multiple choice test online.
 You receive an automated score on the test.
 Class is over.

 You take an online class.
 You “meet” your fellow students on the d-board.
 You read some materials. You find and share some materials too.
 You participate in some discussions of course concepts.
 You take a multiple choice test.
 You receive an automated score on the test.
 Class is over.

 No interaction with peers.
 Students don’t feel “missed” if they don’t participate.
 Not clear why course is online (except perhaps for media elements).
 Potential for immediate feedback is nice -but assessment format is limiting.

 Interaction with peers is great. Serves as a motivator.
 Community is likely to develop.
 Students will feel involved and important if they share examples and resources.
 Assessment format may not be well aligned given the activities.
 Class lacks closure in a manner appropriate to the activities.

 Course objectives, activities, and assessments should be in alignment

This tends to be an issue in courses regardless of medium.
 Example:

In class students conduct a debate

Students are tested on their ability to recall facts

 A not-uncommon scenario

Discussion is used as a learning activity


Students are required to participate
Participation is noted by how many messages were composed by a student
 But does this method measure learning?

 Instructor perspective



There’s too much to assess!
I don’t know what activities to assess!
I don’t know if students really are ready for the test!

How do I know the student actually did the work/took the test?

 Student perspective
 If they’re supposed to discuss, why doesn’t that count as part of their grade?
 If they’re just supposed to do something, why does quality matter?
 I just got a number, no feedback.
 I didn’t get participation feedback.

 Students participating in online activities look for feedback to know:


A. the instructor is reading their contributions
B. their participation is valued

C. their participation is adequate, in terms of quality and quantity
 Feedback need not be individualized to be effective

Whole class commentary provided on a regular basis was found to be just as satisfactory from the student point of view (Dennen, 2001)

 Often, online activities go unassessed

“Add-on” syndrome: Adding an online activity to a previously designed class because it sounds like a good idea

 Students are more likely to participate when then know there is impact on their grade

Direct impact: graded on participation
(quality, quantity or both)


Indirect impact: participation should bolster performance on other assessments
Students quickly become aware if an online activity is not related to assessed learning objectives

 Sometimes the wrong things are assessed
 Examples:

Assessing students’ online moderation skills when the course topic/learning objectives have nothing to do with online moderation

Assessing quantity of participation, but not quality
• which, granted, is easier, but encourages sloppy message posting rather than thoughtful learning dialogues

 A useful tool for checking alignment
 Also great to guide your course design!
QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture.

A Taxonomy
For Learning
Teaching and
Assessing
By Anderson
And Krathwohl
QuickTime™ a nd a TIFF (Uncompre ssed ) decomp resso r are need ed to se e th is p icture.

Options…
 Formative or Summative
 Student-led (Self or Peer) or Teacher-led
 Public or Private
 Process or Product
Other issues to consider…
 “Objective” or Interpretive
 Rubric-based or Wholistic

Formative
 Alleviate student anxieties re: expectations

Seem especially high in online classes
 Encourage working toward mastery
 Can be informal
Summative
 Used for student’s grade
 Assumed to be “best” effort

 Paper draft discussion forum

Start a discussion forum for papers-inprogress

Have each student start a thread and post elements of their papers as they complete them (e.g., topic, major claims, research sources)
• A schedule for each element is useful

Monitor and provide feedback

Teacher-led
 “Traditional” assessment
 Most often summative
Student-led
 Students may assess self or peers
 May be formative or summative
 Can greatly relieve instructor burden
 Students reinforce concepts through feedback process

 Self-tests: Use test tool to create self-tests
(multiple choice, true false)


May wish to track student efforts
Can incentivize use (essentially, use as a learning tool)
 Reflection papers: Have students submit brief, focused papers expressing the strengths and assessments of their assignment(s)

 Feedback groups: Assign students in groups to provide formative feedback on projects and papers

Often raises quality of assignments

Need a structure with clear deadlines

Need prompts and models to guide students

May wish to assess feedback process/contributions

 Conference Presentations: Have students
“present” their work and ask questions/provide feedback to others.


In d-board, have students attach papers to messages; post a message with a synopsis; or attach a powerpoint presentation
Each student/team should have their own thread


Feedback should occur during a defined period of time.
May consider allowing students to rate assignments on certain dimensions

Private
 Work is submitted to the teacher only
 Entire burden of feedback is on teacher
 Important if assessing at fact level
Public
 Peers can see each others’ work (either in process or completed)
 Peers may comment on each others’ work
 Often increases quality of work submitted

 Product


The end deliverable
Look for polish, accuracy
 Process


How the student got there
Look for thoughtfulness of approach, intent

 Easy to do

Many technology tools will archive student work/interactions

Students create a document trail in process
 Helps students develop metacognitive knowledge

Instructors structure/model/encourage productive work processes

Students learn how to manage their own work processes

 For the instructor …

Provides formative feedback on course
(e.g., helps gather data about why students have difficulty with product-oriented assessments)

Clarifies who is doing most work in small group assignments

Helps prevent cheating

 For the student …

Typically improves the quality of their products

Helps them develop productive work processes

Puts on a schedule

Shows that you care about individual growth

 From Classroom Assessment Techniques by Angelo & Cross (1993)
 Step 1: Plan

Choose class

Focus on assessable question

Design project to answer question

Assessment Project Cycle [2]
 Step 2: Implement

Teach target lesson

Collect assessment data

Analyze data
 Step 3:

Interpret results

Communicate results

Evaluate assessment project

 How to do online:

Have students each start their own thread and post topic of interest

Peers and instructors give feedback

Students post thesis statements, research sources, etc., with iterations of feedback

Final paper is posted

 Product: the paper
 Process: quality and timeliness of student work from time when paper is assigned
 Process: quality and timeliness of feedback provided to peers
 Process: responsiveness to feedback received from instructor and peers

II. Discussion Assignments
1. Chain of thought

Have students develop a solution to a problem

Have students indicate what led them to a particular conclusion, method or approach

Can be done in a discussion board

2. Theory to Practice

Have students match up theories you are learning about to actual problems

Present students with problems and have them explain what theories they would use to solve these problems and how they would approach it

Debrief the assignment

3. Synthesizer (i.e., wrapper)


Have students take roles being the weekly synthesizer of class discussion
Add a “meta” level in which students narrate their own experiences while reading the weekly discussion

Reflect on how life relates to discussion

 Tools used

Chat: brainstorming ideas, making group decisions, regular way to feel connected (should be archived)

Discussion board: commenting on drafts

E-mail: quick feedback

File exchange: sharing project files

MS Word:
Track changes

Group Project Assessments
 Product: project files that are turned in
 Process: online archive demonstrating




Who contributed what
Who provided peer feedback
Who worked in a timely manner
How collaborative a group was
 Process: peer ratings
 Process: interim instructor consultations

Group Project Assignments
1. Work Plans



Have students develop a plan of work for their project
Make them outline topic, schedule, resources needed, division of labor and anticipated form of final deliverables
At end of project, have students evaluate how well they followed their own plan and how useful it was

2. Research Trail

Have students document the steps they took in the research process and the results

Ask for a brief reflection on how effective their process was and what they might change the next time

3. Process Presentations

Have students focus on their process as well as their product in class presentations

To maintain focus, ask them to share 3 main lessons learned

Might ask for some process documents to be shared, like an early draft

4. Design Journal

Have students maintain a journal of all ideas related to their project

Encourage sketches, lists, organizational charts, etc.

Require journals to be turned in with final projects

 Have students keep a weekly journal of their thoughts on readings and course content AND real-world related instances that they noticed
 May make these public, with each student having their own discussion thread

 Learners need to see that process is valuable:



Model appropriate processes
Provide students with scaffolding (guide sheets) to structure their processes
Give students feedback on their process


Require students to reflect on their processes
Grade students on process

 Before you assign it, ask yourself “can I reasonably assess it?”
 Rely on students/peers for providing some feedback
 Let students know what to expect up front
 Choose the right tool for the job

Get to know the editing and commenting features of your favorite programs
 Use rubrics!

I make them in Word, and then while grading I highlight or bold the section that applies to the student’s projects. Add a few comments at the bottom
= speedy grading!