Session 5
Assessment in Online Courses
• Kadir—can you find anything about how it is
different? From face-to-face?
• Cheating and online students, what else? How
to handle cheating?
Assessment in Online Courses
Palloff & Pratt (2009) present a list of principles for effective online
assessment. Here are few of those principles:
• Design learner-centered assessments that include self-reflection
• Design and include grading rubrics for the assessment of contributions to
discussions, as well as for assignments, projects and collaboration itself
• Include collaborative assessments through public posting of papers
• Encourage students to develop skills in providing feedback by modeling
what is expected
• Design assessments that are clear, easy to understand, and likely to work
in the online environment
• If you have any group or peer work that results in a grade then must
incorporate a feedback or reflection from to ensure accountability
E-book available in our BB course.
Palloff, R. & Pratt, K. (2009). The assessment and evaluation toolkit. In Assessing the online
learner: Resources and strategies for faculty (pp. 66-135). San Francisco, CA: Jossey-Bass.
Assessment in Online Courses
Consider the assessment and evaluation techniques that you commonly use or observe.
• Also, if you’ll recall, we talked about a few ideas for the prompt feedback principle earlier in the
course (The Key Principles of Quality Online Courses)
• Make learners aware of expectations in advance (e.g. one week for feedback from due date) and
keep them posted (announcement: all projects have been graded).
• Instructors can consider auto-grading options offered by many online course systems. For example,
within the Blackboard Learn system you can create tests that are multiple choice, true/false, or
short answer essays and allow you to set the assessments to automatically provide feedback.
• Incorporate a peer feedback process into your courses through student assignments. They get an
initial level of feedback before turning in the assignment, prompt feedback, and you get a better
assignment in the end.
•
• One particular popular assessment option for online and blended learning is rubrics. In part, their
popularity is based on the level of detail included. “Rubrics help to define the characteristics of a
high-quality assignment and help the student understand assignment and assessment expectations.
Rubrics also provide a range of performance by establishing categories that span the range of
possible outcomes, from basic to exceptional performance on task” (Palloff & Pratt, 2009, p. 33).
Share sample rubrics (566, 575)
Kadir—can you find some sample rubrics for
group work in online courses?
Online Rubric Design Tools
• http://pareonline.net/getvn.asp?v=7&n=25
• http://www.mylearningplan.com/content/products/mlpoasys/differentiat
ed-rubric-design.html
• http://rubistar.4teachers.org/
Evaluation in Online Courses
• Kadir—this topic is yours, keep it brief
• Info from From LDA 5
• Some strategies for evaluating your course
pre, during and post
– Target audience reviewers, readers, SGID:
http://www.purdue.edu/cie/teaching/smallgroup-instructional-diagnosis.html
– mid course evals, post course evals and redesigns
Planning for Continuous Improvement
•
•
•
•
•
The course design process is not static and every course should be evaluated with each delivery.
This is particularly important in the context of online and blended learning as reluctant colleagues
and senior executives often ask: 'Where is the proof that it works?'
Surveys of learner satisfaction, student membership of course committees and scholarly peer
review by colleagues can all contribute to a better understanding of what works and what does not
- and why - in the situated context of your course. To a large extent, a culture of innovation and
excellence in teaching is built on understanding your failures and mistakes as these are the true
source of the insight necessary for continuous improvement.
If you are using a learning management system (LMS) there are additional opportunities to microevaluate. This type of evaluation can occur through polls, reflections on your analysis of online
transcripts, and student activity logs and reports. Notably, you can plan for such forms of evaluation
during the course design process and embed them in the learning architecture.
Arguably, the best way for you to ensure that your course is continuously improved is to embrace a
culture in which the teacher is still a learner. In this sense, continuous improvement and quality
enhancement is an ethos or mindset of always striving to achieve one's personal best performance.
Of course, you are not on your own and keeping in touch with those people with more formal
knowledge of teaching is important. Also being an active participant of a wider professional
community in which there is a nexus between teaching and research will help to expose you to new
and fresh ideas.
Selecting Media
Consider the following two steps for helping you select the appropriate media.
•
•
Your first step will be to consider your learning outcomes. Do they ask for students to be immersed in a realistic experience? How realistic does it need to be in order to be considered
effective?
•
For example. A pre-service teacher learning about classroom management strategies. You could have them observe in real time an actual classroom (webcams), review videos of an
actual classroom (embedded video clips or you tube), engage with a software program that ask them to provide ongoing observations and strategies, or read several scenarios (textbased). Classroom management is one of the most difficult aspects for pre-service teachers to grasp so the more realistic the better. But this brings us to step 2.
•
•
•
•
Your second step involves assessing available technologies. Consider what is available at your institution, what is available online, what is free and what has costs. Are the costs
prohibitive or impractical? Is it a better return on investment (ROI) in terms of time money and other resources to create your own media?
Returning to our classroom management example, it may be difficult to schedule live sessions of actual classrooms. However, you may have several clips that demonstrate your points.
An added benefit is that everyone will be viewing the same clips, so activities can focus on them in particular. This may be enough. Or, you may decide to incorporate some scenariobased software that asks students to work out strategies on their own. The video clips may be used early on in the class while the scenario-based software is used later to demonstrate
growth.
Obviously this 2 step process is but a simple overview, one that will hopefully provide you some guidance nonetheless.
Blending learning note: If you are considering selecting media for a blended course you may find the above example useful. However, you should also consider than blended implies
some face-to-face class time. Given this other options may also be suitable, two-way interactive video(webcam) where your class can observe a live classroom together or on site visits to
a live classroom which may be easier to schedule if location specific.
•
Following are some general ideas for consideration when selecting media:
–
–
–
–
–
–
–
–
•
Always remember, in order to successfully implement any media you need to focus on what you want your students to be doing, and how you are going to measure their progress and success. Just
because social media is all the buzz doesn’t mean it belongs in your course.
Another important factor is to determine whether you are designing individual or group instruction, is the media easy to use for groups communication, editing, etc.
You are encouraged to take risks with media and strategies. Let your creative juices flow!
Just the same, it is very important you consider the ROI (return on investment) for both development and implementation in terms of time, costs, skills, and resources available.
Keep in mind that the Media Choices should be weighed against "ideal" media, taking into account time and resources—especially the issue of student access to the media and necessary components
(e.g. downloading times, need for microphones, need for plug-ins).
Is the media more useful as a tool for ‘Content delivery’, ‘Interaction’ or ‘Assessment’?
What is your level of skill with a particular media? Are you willing to step outside of your comfort zone?
Don’t be afraid to ask for help. Ask students, colleagues IT and media specialists for assistance, training and resources.
Final Tips for a Successful Course
•
Is the course ready to begin?
Plan to have your course ready at least one month in advance. Simonson, et. al. (2012) suggest you consider beginning your work on the course 3-5
months in advance.
•
Is everything in working order?
Test everything! Moreover, remember to create everything in more than one format – word instead of PDF, for example (think accessibility!) – and to
use multi-platform technologies (check that your videos run on both Macs and PCs!).
•
Is there consistency in your course?
Consistency is key in the design. This does not mean that your course modules need to be identical, but that they have the same layout and tools, and
that pathways are intuitive. This way students can easily find what is expected and due each week.
•
•
•
Is there a variety of learning activities?
Diversifying course learning activities keeps learners engaged at multiple levels and can also speak to the strengths and diversity of your students.
Have you set learners enough time to develop peer-to-peer activities?
Allow adequate time in your course schedule for the development of online discussions; a minimum of one week for each discussion is recommended
(Conrad & Donaldson, 2004, p. 19).
Have you made it clear how learners should work through the course?
Clear directions should be provided for what learners should do at every stage of the course (Simonson, et. al. 2012, p. 176).
•
Can you regularly check-in with students and personalize their learning experience?
Provide a space for a ‘Weekly Overview’ (if your course is set up by weeks), to include highlights (text, audio or video) of the weekly readings.
Additionally, use this as a place to personalize the courses.
•
When can learners access you?
Establish a communication framework. Let learners know when you will be in the course to check for updates and questions, how long it will take you to
respond to emails, and when you will not be available. Also, establish an ‘Ask the teacher, ask a peer’ discussion thread.
See also St. Germain, 2011. Five Common Pitfalls of Online Course Design:
Available online at http://www.facultyfocus.com/articles/online-education/five-common-pitfalls-of-online-course-design/)
Sample Media Showcase:
Respondus Lockdown and Monitor