Veronica Diaz, PhD, drvdiaz@gmail.com
DAY 2: DESIGNING AND
DEVELOPING A BLENDED COURSE
Discussion
• Mapping the course
• Teaching Online Survey
Today’s Agenda
• Building community and collaboration
– Online discussions
•
•
•
•
•
Student teams
Assessment and assessment techniques
Academic integrity
Copyright issues
Open content
BUILDING COMMUNITY AMONG
STUDENTS
4
Jane Livingston, 2006, Building Community in a Blended Course, Educause
5
What makes a successful community?
•
•
•
•
•
•
•
Individuals feel safe
Get questions answered
Have conversations
Get resources/information
Support
Know eachother
Produce a product
6
Building Community
• Start early, F2F
• Make it relevant
• Create opportunities for
engagement
• Encourage participation
7
Collaboration Benefits
• Passive to interactive
• Increase retention of
class materials
• Develops critical
thinking skills
• Knowledge construction
•
•
•
•
Builds community
Team building
Interpersonal skills
Importance of
emphasizing the
relationship of
interactive activities to
“content”
Ross Mayfield: http://ross.typepad.com/blog/2006/04/power_law_of_pa.html
In Class: Student Collaborative or
Interactive Activities
• In class writing activities
• 5 minute discussion
questions
• Scripted scenarios for
role playing
• Think-Pair-Share
•
•
•
•
Note Check
Case Studies
Discussions, Part I
Group Projects
• More examples on CD
under COLL
Online: Student Collaborative or
Interactive Activities
•
•
•
•
•
Case studies
Discussions, Part II
Forums: Panel or Symposium
Experiential Learning
Group Projects
–
–
–
–
Role-play
Games & Simulations
Demonstrations
Online Presentations
• http://www.ion.illinois.edu/resources/tutorials/pedagogy/i
nstructionalstrategies.asp
• http://www.ion.illinois.edu/resources/otai/
11
FACILITATING AND ASSESSING
ONLINE DISCUSSIONS
In Chat
• How do you use
classroom discussion in
your current courses?
• How do you assess
students?
• Netiquette:
– http://www.albion.com/netiquette/c
orerules.html
CONSIDER: DISCUSSION BOARDS BE
GRADED WITH SUBSTANTIAL POINTS
ASSIGNED
Ways to Use a Discussion Board
• Prepare for upcoming
in-class discussion (preassignment)
– Reading
– Review of literature
• Follow-up to in-class
discussion (continue
discussion or postassignment)
Source: Teaching Online A Practical Guide by Ko and Rossen
• Extension of in-class
discussion and
assignments
(exploratory, will not be
covered in class)
Using a Discussion Board
• Question and answer forum
(to create an FAQ page)
• Pose a problem and have
students generate possible
solutions – discuss those
solutions
• Students post homework or
projects and get classmate
feedback
• Case study
• Allow students to self assign
groups that will take charge of
a particular week’s discussion,
facilitation, and summary
Source: Teaching Online A Practical Guide by Ko and Rossen
Using a Discussion Board
• Students critique
classmates’ work using
provided evaluation
guidelines
• Find/evaluate web
resources on
lesson/topic and discuss
results
• Invite guest
speakers/lecturers
(student led)
• Debate/negotiate about
Questioning Techniques
Before
“Name and describe three
social systems theories that
apply to community
development.”
After
“What theory of community
development did you find
yourself relating to most?
Why? How would you apply
that theory to our learning
community?”
Lessons from the Cyberspace Classroom: The Realities of Online Teaching, Rena Palloff and Keith Pratt (pg. 121)
Moderating and Facilitating
Online Discussion
•
•
•
•
•
•
Encourage participation
Ensure that some students don’t dominate
Keep discussion focused
Bring out multiple perspectives
Summarize highlights
Do not dominate or be over-involved in the
discussion
• Assign and rotate roles for students
Source: Gregg Kearsley Online Education: Learning and Teaching in Cyberspace, Wadsworth: 2000, p. 85
Source: Tom Nolan, Sonoma State University http://www.sonoma.edu/users/n/nolan/501/powerpoint/moderating_facilitating/
Discussion Resources
•
•
•
•
•
•
•
•
•
http://www.dartmouth.edu/~webteach/articles/discussion.html
http://www.elearnmag.org/subpage.cfm?section=case_studies&article=35-1
http://www.educause.edu/EDUCAUSE+Quarterly/EDUCAUSEQuarterlyMagazine
Volum/UsingInteractioninOnlineDiscus/157367
http://www.ic.sunysb.edu/Class/che326ff/discussion_board/etiquette.pdf
http://www-usr.rider.edu/~suler/psycyber/extendclass.html
http://www.online-distance-learningeducation.com/article_info.php/articles_id/51
http://fdc.fullerton.edu/catalog/blackboard/bbb3050_discussionboard.htm
http://docs.google.com/gview?a=v&q=cache:wCGe35K0wGkJ:www.mtsu.edu/~
webctsup/faculty/manual/WebCT_DiscussionBoard_TipsPedagogy.pdf+discussion+board+rubric&hl=en&gl=us
http://74.125.95.132/search?q=cache:kpTuNOYdLi4J:www.tulsacc.edu/dl/faculty
/bb_tips/discussion_rubric1.doc+discussion+board+rubric&cd=1&hl=en&ct=clnk
&gl=us&client=safari
Use of Discussion Boards?
• How will you use the
discussion board?
• Communication and
collaboration
resources on CD
STUDENT TEAMS AND
BLENDED LEARNING
22
Which best describes your
experience with student teams?
[do in chat]
23
Using Teams
• Based on the work of Larry
Michaelsen (University of
Central Missouri)
• http://teambasedlearning.ap
sc.ubc.ca/
• 3 Keys
– Promoting ongoing
accountability
– Using linked and mutually
reinforcing assignments
– Adopting practices that
stimulate idea exchange
24
Promoting Ongoing Accountability
• Require pre-group work
• Require group members to express
individual opinions and monitor via
another member
• Include peer evaluation in grading
• Readiness Assurance Process
– Test over readings
– Group: Test, discuss, reach consensus and retest
– Provide information for peer feedback process
25
Using Linked and
Mutually Reinforcing Assignments
26
Adopting Practices that
Stimulate Idea Exchange
Use of assignments that
create conditions that foster
give-and-take interaction
• Assign roles
• Use permanent groups
• Allow some in-class group
work
• Size: 4-7
Diversity of opinion, ideas,
and perspectives
• Not too easy
• Not too much writing
• Employ, select, apply
concepts from the course
Team Teaching Tips
•
•
•
•
•
Outline learning goals
Teach team skills
Clear and detailed instructions
Rubric
Stages of team development
–
–
–
–
Forming - polite but untrusting
Storming - testing others
Norming - valuing other types
Performing - flexibility from
trust
28
Team Contracts
•
•
•
•
•
•
•
•
•
Purpose, goals, and missions
Expectations
Roles
Conflict resolution strategies
Meetings
Communication
Decision-making policy
Agendas
Record-keeping
29
Other Resources
• Team Based Learning
(Michaelsen)
– http://teambasedlearning.a
psc.ubc.ca/?page_id=9
• Video Demonstrations
– http://teambasedlearning.a
psc.ubc.ca/v/michaelsenvid.
html
30
31
4 Questions as Guides
• What do I want students to be
able to DO after this unit of
instruction (behavioral outcomes)
• What will students have to KNOW
to do XYZ (learning outcomes)
• How can I ASSESS whether or not
students have successfully
mastered key course concepts?
• How can I tell if students will be
able to USE their knowledge of
key course concepts? (application)
32
In Chat
• How might the team
based learning
approach or parts of
TBL fit into blended
learning?
33
In Chat: Select the 2 most
common assessments you’ve used
•
•
•
•
Multiple choice tests
Written exams
Problem-based projects
Research papers
•
•
•
•
Case studies
Short answer
Large projects
Essay questions
Assessment
and
Learning
Instructor delivers the content
Instructor provides opportunities for
additional time on task
Learners receive feedback on their
application
McGee, Patricia, 2009
Learners interact with the
content/instructor/learners/resources
Learners apply and practice the
concepts
Purposes of Assessment
Next steps in
learning
Next steps in
teaching
Status of course
Documentation
of Achievement
assessment
rubrics
practice
examples
demonstration/introduction
What Students Know
transfer of learning
What students can do
Facts
Procedures
Principles
Strategies
Critical thinking
Problem finding/solving
Creative thinking
Typically measured by
objective assessments
Typically measured by
performance assessments
Homework, quizzes, tests,
exams
Projects, cases, problems,
designs, experiments
Monitor and Adjust
Objectives?
Outcomes?
3
Online Assessment Principles
•
•
•
•
Match learning objectives with assessments
Take precautions to limit the possibility of cheating
Communicate assessment tasks clearly
Use formative assessment to promote deeper learning;
consider alternative forms of assessment such as
portfolios
• Use self-assessments to improve learning and selfawareness
• Have students conduct peer-assessments (may be
particularly effective when used in conjunction with
group work)
Source: http://www.sjc.cc.nm.us/pages/2852.asp
Sound Assessment & Measurement
• Assessments selected measure the stated
learning objectives
• Grading policy
• Process of evaluation of students’ work is
clear
• Selected assessment instruments
• Practice
• Feedback
Assessment Development
• Specify educational objectives/intended
outcomes.
• Select assessment measures and techniques.
• Specify assessment criteria.
• Evaluate student performance on
exams/projects for course grades.
• Evaluate student performance on course
measures to assess effectiveness.
[respond in chat]
WHAT OTHER PRINCIPLES HAVE
YOU APPLIED TO ASSESSMENT
DEVELOPMENT?
Role of Objectives
• Review course objectives
– Ask, are these for skills, knowledge, or abilities
• Are they measurable
– Demonstrate (authentic) or
– Reflect on knowledge/experience
• What do they say about student outcomes
• What is the goal of assessment, what do you want the
student to be able to do or demonstrate they can do
– Use verbs to describe this
– Share with students
Objective ABCDs
• Audience (the learners)
– Identify who it is that
will be doing the
performance (not the
instructor).
• Behavior
(Performance):
– Make sure it is
something that can be
seen or heard.
• Condition (under which
the learners must
demonstrate their
mastery of the objective):
– What will the learners be
allowed to use? What
won't the learners be
allowed to use?
• Degree (HOW WELL the
behavior must be done):
– Common degrees include:
Speed, Accuracy, Quality
Goals
Assessment
Selection
Objectives
• Example: recall of factual knowledge
– Use multiple choice or short response assessment
• Example: complex learning outcomes, such as
reasoning, communication, teamwork
– a performance assessment is likely to be
appropriate
What needs to be assessed?
•
•
•
•
Course goals
Module objectives
Activity objectives
Content Knowledge:
declarative, intellectual
skills (concept,
principle, procedure,
problem solving)
• Thinking Skills
• Inter/Intrapersonal
Skills: collaboration,
cooperation
• Technology
Literacy/Proficiency
• Interaction
• Process
• Other
Performance vs
Objective Assessments
Learning
can be
assessed…
• Via another activity
–
–
–
–
–
Cases
Projects
Presentations
Debates/negotiations
Research paper
• Be the activity itself
– Quiz
– Multiple-choice test
– Simulation
Alignment: Do your…
•
•
•
•
Learning objectives
Resource and materials
Learner interaction and activities
Course technologies
Support student performance?
Assessment Alignment
• Objective is to “design a research study
related to a professional field,” but the
assessment is a short answer test
• Objective is to “understand particular
statistical methods for various types of
research,” but the assessment is to write an
essay
WHAT SHOULD GO INTO AN
ASSESSMENT TOOLKIT?
Assessment Toolkit
What needs to be
assessed?
How will the
learner be
informed about
progress?
What is the
function of the
assessment?
What is the
appropriate
measure?
What criteria will
be used to assess?
What tools offer
the greatest
affordance for the
measure?
What strategies
can be used to
assess?
P. McGee
Learners should know….
• That an assessment is coming
– A word about helping students plan
• What happens after the assessment
• How to prepare for an assessment
• What participating in the assessment will look
like
• Practice
Suggestions for Better Assessment
•
•
•
•
•
•
Build multiple “check point” assessments
Allow open note/book timed tests
Have in class quizzes and tests as well
Project-based assessments
Portfolio assessments
Assign group work in a wiki area that tracks
student participation
• Provide best practice guidelines for taking exams
online
For example
Chat – content learning
Chat - performance
• Answer questions correctly
• Ask questions that
• Participate throughout
discussion
• Do not interrupt
• Respond when called upon
• When appropriate, come to
chat prepared.
– Helped others learn
– Extended discussion
• Make contributions to
discussion that extended
materials
Self-Assessment
•
•
•
•
•
Techno CATs
Before/after unit
Reflection
Feedback on design
Feedback on technology
•
•
•
•
•
Increases accountability
Engages students
Starts discussions
Early alert
Practice
1.
What was the one most useful thing you learned in this
assignment, unit or module?
2.
What suggestions would you give other students on ways to
get the most out of this assignment, unit or module?
3.
In what area did you learn or understand the most? Least?
4.
List three ways you think you have developed or grown as a
result of this assignment, unit or module?
5.
What did you learn about writing, research, (or any other skill)
from this assignment, unit or module?
6.
What problems did you encounter in this assignment, project,
unit, or tool that was used?
7.
What unit/module of this course was your best work and why?
Examples
CATs: What are they?
• A method used to
inform you on ….
– Students learning
– Effectiveness of course
content
– Effectiveness of
teaching methods
CAT Benefits
•
•
•
•
•
Learner-centered
Teacher-prompted
Mutually beneficial
Formative
Fast to administer and
interpret
• Non threatening
• Ongoing
• Foster trust between
student and instructor
Basic CAT Steps
1.
Choose a learning goal to
assess
2.
Choose an assessment
technique
3.
Apply the technique
4.
Analyze the data and share
the results with students
5.
Respond to the data, i.e.,
make modifications as
necessary
5 Suggestions for CATs
• Customize to your specific needs
and learning environment
• Should be consistent with your
instructional philosophy
• Test out a CAT and assess their
effectiveness
• Allow extra time to carry out and
respond to the assessment
• Let students know what you learn
from their feedback and how you
and they can use that information to
improve learning
CAT Examples
•
•
•
•
Chain notes
One-sentence summary
Application cards
Student-generated test
questions
• Can be easily modified or
converted to an online
environment
CATS as Formative Assessments
• CATs (most can be easily converted)
• Other ideas: http://pareonline.net/getvn.asp?v=9&n=7
• Teaching Tips:
http://honolulu.hawaii.edu/intranet/committees/FacD
evCom/guidebk/teachtip/teachtip.htm#assessment
• Polls (polldaddy.com, survey monkey, survey tool in
CMS)
• Drill & practice activities with participation points
(online quizzes)
• Peer review & feedback
Assessment Resources
• http://honolulu.hawaii.edu/intranet/committees
/FacDevCom/guidebk/teachtip/assess-2.htm
• http://www.ntlf.com/html/lib/bib/assess.htm
• http://www.ntlf.com/html/sf/vc75.htm
• http://technologysource.org/article/classroom_a
ssessment_techniques_in_asynchronous_learnin
g_networks/
• http://www4.nau.edu/assessment/main/researc
h/webtools.htm
Exercise
• Review the CATs and pick one
• Identify the goal for your CAT
• Explain why this CAT is
helpful/necessary in this
particular area of the course
• How and when will students
receive feedback on the CAT
Rubrics
• Specifically state the criteria
for evaluating student work
• Are more specific, detailed,
and disaggregated than a
grade and can help students
to succeed before a final
grade
• Can be created from
– Language in assignments
– Comments on students’
papers, or
– Handouts intended to help
students complete an
assignment
Development Steps
• Identify what you are assessing (e.g., critical
thinking, writing, process, participation)
• Identify the characteristics/behavior of what
you are assessing (e.g., presenting,
problem-solving)
• Decide what kind of scales you will use to
score the rubric (e.g. checklists, numerical,
qualitative, or numerical-qualitative)
• Describe the best work you could expect
using these characteristics: top category
• Describe the worst acceptable product using
these characteristics: lowest category
More Steps
• Develop descriptions of intermediatelevel products and assign them to
intermediate categories:
– 1-5: unacceptable, marginal,
acceptable, good, outstanding
– 1-5: novice, competent, exemplary
– Other meaningful set
• Test it out with colleagues or students
by applying it to some products or
behaviors and revise as needed to
eliminate ambiguities
Rubric Tips
• Develop the rubric with your students
• Use same rubric that was used to grade
• Use examples to share with students, so
they can begin to understand what
excellent, good, and poor work looks like
• Have students grade sample products
using a rubric to help them understand
how they are applied
• In a peer-review process, have students
apply the rubric to eachother’s work
before submitting it for official grading
Benefits
• Allows assessment to be
more objective and
consistent
• Focuses instruction to clarify
criteria in specific terms
• Clearly shows the student
how their work will be
evaluated and what is
expected
• Promotes student awareness
of about the criteria to use in
assessing peer evaluation
Rubric Resources
•
•
•
•
•
•
•
•
•
•
•
http://www.uwstout.edu/soe/profdev/rubrics.shtml
http://www.calstate.edu/AcadAff/SLOA/links/rubrics.shtml
http://www.iuk.edu/~koctla/assessment/rubrics.shtml
http://www.csupomona.edu/~uwc/faculty/CSU-EPTScoringGuide.shtml
http://condor.depaul.edu/~tla/html/assessment_resources.html
http://www.winona.edu/AIR/rubrics.htm
http://www.engin.umich.edu/teaching/assess_and_improve/handbook/di
rect/rubric.html
http://www.seattleu.edu/assessment/rubrics.asp
http://wsuctproject.wsu.edu/ctr.htm
http://www.utexas.edu/academic/diia/assessment/iar/students/report/ru
brics-types.php
Rubric Template:
http://edweb.sdsu.edu/triton/july/rubrics/Rubric_Template.html
Activity
• Consider how you
might use or modify
one of these for your
course
• Share in chat
Academic Integrity
In the networked/information age
What’s Changed?
CHANGED
• Cheating mechanisms
• Lines have blurred
– Collaboration
• Ease of cheating
• Ease of monitoring cheating
• Ease of preventing cheating
NOT CHANGED
• Definitions
• Importance of informing,
educating, and enforcing
• Honor code policies
and procedures
• Student assessment quality,
validity, reliability
What does it look like today?
• Having someone edit
students' papers
(grammar, style, spelling)
• Note-taking services
• Sharing password to
course management
systems
• Submitting a paper from a
term paper service
• Working on a graded
assignment together
• Revising a paper that was
found on the internet
• Using a cell phone (IM) to
transmit exam
information
• Looking at another
student's work while
taking an exam
• Sharing computers
• Public computers
• Other?
Cheating and Plagiarism Websites
•
•
•
•
http://www.cheathouse.com/
http://www.schoolsucks.com/
http://www.researchpaper.com/
http://www.allpapers.com/intro.htm
Pedagogical Solutions
• Assign work and tests that
are due frequently
throughout the semester
• Assign work that builds
sequentially on prior
submitted work, such as
revisions of drafts
• Administer unannounced
quizzes or participation
• Take-home tests/quizzes
Pedagogical Solutions
• Require assignment and test
responses to relate the
subject matter to students'
lived experiences or test
questions on current events
• Meet with students
individually (online/F2F) and
test/quiz them on course
content
• Review students’ previous
work
Pedagogical Solutions
• Debrief/interview a student
concerning their test/quiz
asking specific questions about
their answers
• Use alternative modes of
student assessment such as
portfolios, rubrics, selfassessment, peer assessment,
and contracts
• Use multiple methods of
measuring performance
• Use application-type exams
(PBL, case based learning)
• Define academic integrity as a class
• Encourage students to come to you if they
are confused about citation practices
• Be a good role model; cite sources in your
lectures
• Talk about academic honesty with your
students, and make sure they understand
both the reasons and the tools for avoiding
plagiarism
• Contracts for integrity
1. Affirm the importance of
academic integrity—
workplace standards.
6. Reduce opportunities to
engage in academic
dishonesty.
6. Encourage student
responsibility for academic
integrity.
7. Challenge academic
dishonesty when it occurs
and make it public.
7. Clarify expectations for
students.
8. Help define and support
campus-wide academic
integrity standards.
8. Develop diverse forms of
assessment.
Alternative Means of Evaluating
Student Performance
•
•
•
•
•
•
Center for Academic Integrity
Assessment and Evaluation for Online Courses
Authentic Assessment Resource Site
Alternative Assessment and Technology (ERIC Digest)
Classroom Assessment Techniques
Virtual Academic Integrity Laboratory
In Chat
• Identify one academic
integrity challenge you
have or may experience
in the future (in a
blended environment)
• Identify a potential
solution
Questions and Answers
Next Session
• When: Friday, September 04, 2009, 2:30 PM5:30 PM
• Where:
https://www2.gotomeeting.com/join/1424953
942
– Use your microphone and speakers (VoIP) - a headset
is recommended.
– Or, call in using your telephone. Dial 218-844-4926
– Access Code: 142-495-394
– Audio PIN: Shown after joining the meeting
– Meeting ID: 142-495-394
Contact Information
Veronica M. Diaz, PhD
drvdiaz@gmail.com
http://www.mcli.dist.maricopa.edu/diaz/
Copyright Veronica Diaz, 2009. This work is the
intellectual property of the author. Permission is granted
for this material to be shared for non-commercial,
educational purposes, provided that this copyright
statement appears on the reproduced materials and
notice is given that the copying is by permission of the
author. To disseminate otherwise or to republish requires
written permission from the author.