Teaching sustainable online
research practices across
the curriculum:
The Q6C Solution
Sarah Read (English)
Kate Deibel (CS / Education)
Tim Wright (History)
University of Washington
Computers & Writing 2009
 Goals:
 Transcend traditional boundaries
 Academic disciplines
 K-12 and higher education
 Online and offline
 Organizers, attendees, and presenters
 School, work, and play
 Develop a sustainable perspective on lifelong
computing and communication
Computers & Writing 2009: Ubiquitous and Sustainable Computing
2
Today’s Plan
 Framing Q6C Within Conference Intent
 A Taste of Application
 What is Q6C?
 Development
 Overview
 Using Q6C
 For the Teacher
 For the Students (and Teacher)
 Conclusion
 Discussion and workshop
Computers & Writing 2009: Ubiquitous and Sustainable Computing
3
The Issue
Computers & Writing 2009: Ubiquitous and Sustainable Computing
4
The Research Scenario
You are in a college geology course and have
been assigned to brief the class on the
Crandall Canyon Mine disaster.
Answer the question:
Was the cause of the mine collapse
geological, and what was it?
Computers & Writing 2009: Ubiquitous and Sustainable Computing
5
First Hit: Why is this not a good source?
Authoritative
Source and
Author
The
Answer
Questions raised by Q6C
 How do researchers know when to stop
looking for sources?
 What do experienced researchers in a field
know about when to stop that newcomers
don’t know?
 How can newcomers to a field be
supported to do research with the savvy of
experienced researchers?
Computers & Writing 2009: Ubiquitous and Sustainable Computing
7
Q6C: Applied to history
Computers & Writing 2009: Ubiquitous and Sustainable Computing
8
Symptoms of a Serious Problem
In regards to online sources, students...
 …stop too soon when evaluating a source
 …fail to keep a skeptical frame of mind
 …tend to focus on surface/superficial details
 …trust the top search results
 …reject Wikipedia outright
 …accept Wikipedia outright
 …fail to transfer critical skills across domains
Computers & Writing 2009: Ubiquitous and Sustainable Computing
9
An Information Literacy Skill
 The Goal:
Teach students to be savvy, critical
consumers of Internet sources for both
academic and non-academic purposes
 Challenges for Teaching
 What skills should be taught?
 How can those skills best be conveyed?
 What is needed to promote skill transfer
across disciplines / outside of the classroom?
Computers & Writing 2009: Ubiquitous and Sustainable Computing
10
Today’s Plan
 Framing Q6C Within Conference Intent
 A Taste of Application
 What is Q6C?
 Development
 Overview
 Using Q6C
 For the Teacher
 For the Students (and Teacher)
 Conclusion
 Discussion and workshop
Computers & Writing 2009: Ubiquitous and Sustainable Computing
11
Previous Work and Research
McCarthy
(1987)
Different rules and expectations for
student writing exist across disciplines
Sorapure et al.
(1998)
Internet poses challenges and
opportunities for student research
Kvavik (2005)
Students report IT confidence but
overestimate source judging abilities
Head &
Students report difficulties and
Eisenberg (2009) overload in conducting research
Many authors
Checklists for evaluating reliability of
Internet sources
Computers & Writing 2009: Ubiquitous and Sustainable Computing
12
The Checklist
Authorship
 Is there an author? You may need to…
 Can you tell whether the author is
knowledgeable and credible? If the
author's qualifications aren't listed…
Sponsorship
 What does the URL tell you? The URL
ending often specifies the type of group
hosting the site: commercial (.com),
educational (.edu), nonprofit (.org), …
Currency
 How current is the site?
 How current are the site's links? If many of
the links no longer work, the site may be
too dated for your purposes.
Excerpt from Hacker’s A Pocket Manual of Style (2008)
Computers & Writing 2009: Ubiquitous and Sustainable Computing
13
Pitfalls of the Checklist
Authorship
 Is there an author? You may need to…
 Can you tell whether the author is
knowledgeable and credible? If the
author's qualifications aren't listed…
Sponsorship
 What does the URL tell you? The URL
ending often specifies the type of group
hosting the site: commercial (.com),
educational (.edu), nonprofit (.org)…
Currency
 How current is the site?
 How current are the site's links? If many of
the links no longer work, the site may be
too dated for your purposes.
INACCURATE: .org has
never been restricted
to only nonprofits
Excerpt from Hacker’s A Pocket Manual of Style (2008)
Computers & Writing 2009: Ubiquitous and Sustainable Computing
14
Pitfalls of the Checklist
Authorship
 Is there an author? You may need to…
 Can you tell whether the author is
knowledgeable and credible? If the
author's qualifications aren't listed…
Sponsorship
 What does the URL tell you? The URL
ending often specifies the type of group
hosting the site: commercial (.com),
educational (.edu), nonprofit (.org)…
Currency
 How current is the site?
 How current are the site's links? If many of
the links no longer work, the site may be
too dated for your purposes.
Not all domains are
regulated
Domains reflect only
general purposes and
not specific pages
Excerpt from Hacker’s A Pocket Manual of Style (2008)
Computers & Writing 2009: Ubiquitous and Sustainable Computing
15
Pitfalls of the Checklist
Authorship
 Is there an author? You may need to…
 Can you tell whether the author is
knowledgeable and credible? If the
author's qualifications aren't listed…
Sponsorship
 What does the URL tell you? The URL
ending often specifies the type of group
hosting the site: commercial (.com),
educational (.edu), nonprofit (.org)…
Currency
 How current is the site?
 How current are the site's links? If many of
the links no longer work, the site may be
too dated for your purposes.
Ignores complexity of
web authorship
Encourages the usage
of titles, degrees, and
symbols of authority to
determine credibility
Excerpt from Hacker’s A Pocket Manual of Style (2008)
Computers & Writing 2009: Ubiquitous and Sustainable Computing
16
Pitfalls of the Checklist
Authorship
 Is there an author? You may need to…
 Can you tell whether the author is
knowledgeable and credible? If the
author's qualifications aren't listed…
Sponsorship
 What does the URL tell you? The URL
ending often specifies the type of group
hosting the site: commercial (.com),
educational (.edu), nonprofit (.org)…
Currency
 How current is the site?
 How current are the site's links? If many of
the links no longer work, the site may be
too dated for your purposes.
Suggests recent data
as being more reliable
Update frequency will
vary by the type of site
Excerpt from Hacker’s A Pocket Manual of Style (2008)
Computers & Writing 2009: Ubiquitous and Sustainable Computing
17
Criticisms of the Checklist
 Inherent problems
 Emphasis on surface features over content
 Simplistic yes/no questions with no guidance
 Erroneous indicators of credibility
 Students fail to develop information literacy skills
and critical practices
 Meola (2004)
 Helms-Park & Stapleton (2006)
 Need for better evaluative methods to develop
sustained, transferable skills
 Sidler (2002)
Computers & Writing 2009: Ubiquitous and Sustainable Computing
18
Developing Q6C
 Intent:
 Develop tools to support instructors in teaching web
literacy skills
 Guiding Principles:
 Emphasize the process of source evaluation, not the
end product
 Recognize disciplinary differences
 Promote student metacognition and transfer of skills
across and outside academia
 Approach:
 Discuss source evaluation across the curriculum
 Develop a working model of the evaluation process
Computers & Writing 2009: Ubiquitous and Sustainable Computing
19
Discussions about Source Evaluation
Cross-disciplinary discussions: Sarah, Kate and Tim
Research Literature














Ayers (2006)
Britt & Aglinskas (2002)
Head & Eisenberg (2009)
Helms-Park & Stapleton (2006)
Hunt & Hunt (2006)
Kvavik (2005)
Lorenzo & Dziuban (2006)
Meola (2004)
Oblinger & Oblinger (2005)
Sidler (2002)
Sorapure et al. (1998)
Sullivan and Porter (1997)
Thompson (2003)
Wineburg (1991,1991,1999)
Source Evaluation Tools










Barker & Kupersmith (2009) *
Beck (2009) *
Cohen & Jacobson (2009) *
CARS / CAFÉ – Harris (2007) *
Big6 – Eisenberg & Berkowitz (2001)
Hacker (2008) *
TAP – Johnson & Lamb (2007) *
CSU How to… – Lederer *
Rampolla (2007) *
Smith (1997) *
Computers & Writing 2009: Ubiquitous and Sustainable Computing
* Checklist variant
20
Q6C: Modeling Source Evaluation
Question
Categorize
Critique Rhetorically
Characterize Authorship
Contextualize
Corroborate
Conclude
Computers & Writing 2009: Ubiquitous and Sustainable Computing
21
Q6C: The Start
Question
 Maintain a skeptical
frame of mind
 Ask questions
Critique Rhetorically
relevant to your
research, purpose,
Characterize Authorship
and discipline
Contextualize
Categorize
Corroborate
Conclude
Computers & Writing 2009: Ubiquitous and Sustainable Computing
22
Q6C: The End
Question
 Is the source
credible?
Categorize
 Is the source useful
Critique Rhetorically
for your research
goals?
Characterize Authorship
Contextualize
Corroborate
Conclude
Computers & Writing 2009: Ubiquitous and Sustainable Computing
23
Q6C: The Middle
Question
 Different dialogues
to engage in with a
Categorize
potential source
Critique Rhetorically
 Features and
relative importance
Characterize Authorship
shaped by research
Contextualize
task and discipline
 Not necessary to do
Corroborate
all components nor
Conclude
in any order
Computers & Writing 2009: Ubiquitous and Sustainable Computing
24
Q6C: The Cloud
Question
Categorize
Critique
Rhetorically
Characterize
Authorship
Contextualize
Corroborate
Conclude
Repeat as necessary
Computers & Writing 2009: Ubiquitous and Sustainable Computing
25
Simple Instantiation of Q6C
Topic: Information / opinion on recent U.S. policy
towards net neutrality
Series of posts
on Blogger
Question
Categorize
Most recent post
was 2001
Characterize
Authorship
Computers & Writing 2009: Ubiquitous and Sustainable Computing
Too old to
be useful
Conclude
26
More Complex Instantiation of Q6C
Topic: Information / opinion on recent U.S. policy
towards net neutrality
Many posts on
ScienceBlogs
Question
Likely to be
useful
Conclude
Relevant posts
in 2008-09
Categorize
Characterize
Authorship
He is a Swiss
citizen
Part owner of
a European ISP
Characterize
Authorship
Characterize
Authorship
Computers & Writing 2009: Ubiquitous and Sustainable Computing
Provides links
to citations
Corroborate
Useful but
biases exist
Conclude
27
Summary of Q6C
 Working model of source evaluation process
 Definite start and end points
 Iterative and integrative middle
 Emphasizes research as a process
 Only concerns one element of all of research
 Not specific to particular disciplines
 Domain knowledge, heuristics, and criteria need to be
supplied by instructors
 Provides a common language for discussion
 Dirty little secret… not limited to just online sources
But how do you use it in teaching?
Computers & Writing 2009: Ubiquitous and Sustainable Computing
28
Classroom Applications of Q6C
Two approaches for using Q6C:
1. Using Q6C to inform teaching practice and
assignment and lesson design (Sarah)
2. Infecting students to the practice and habit of
source evaluation via the Q6C model (Tim)
Computers & Writing 2009: Ubiquitous and Sustainable Computing
29
First Hit: Why is this not a good source?
The
Answer
Authoritative
Source and
Author
Why is this not a best source?
Authoritative
Source
Wrong genre?
But current:
07-08
Partial
Answer
Scientific
Source
Where to go from here?
Purpose
tangential
to the
research
question
Is there a
better
source?
Aha! The most credible and the most
useful source!
Directly relevant to
research question
Peer
Reviewed
How to get a student (or a newcomer) to
this “best” source?
Finally, a complete
answer relevant to
the research
scenario
Visualizing the Q6C Process
Question: Was the cause of the mine collapse geological, and what was it?
Scenario: College-level geology course
Government
agency relevant
to issue (.gov)
1st hit: MSHA
web page
Characterize
Authorship
Not a scientific
source
Part of the
story, but not
citable
Contextualize
Conclude
Purpose: tangential
to research question
2nd hit: Berkeley lab
Annual report
.edu /
Scientific lab
Characterize
Authorship
Contextualize
Genre: annual report
(not peer reviewed)
Categorize
BUT: What sources
does it cite?
Corroborate
3rd hit: UT
seismology
scientific report
Best source:
relevant and
credible
Conclude
Student Reflections:
Corroboration
is king (at least
in this research
scenario)
Q6C as a heuristic for
lesson/assignment design
1. Identify a research scenario relevant to course context
2. Identify the tacit research practices of an
experienced researcher in that knowledge domain:
think Q6C.
3. Construct a scaffolded lesson or assignment to
teach explicitly the domain knowledge necessary for
critical source evaluation.
4. Plan for reflective writing or discussion to promote
metacognition about research practices.
OUTCOMES: 1. students locate more useful and credible sources
2. begin to learn how to approach any new research scenario.
Other choices when planning a Q6C
lesson or assignment
 Prepare an archive to anticipate the
research process or set students loose?
 How are the research process and domain
knowledge weighted in assignment
evaluation?
 Explicitly teach the meaning of Q6C terms,
or prompt students to invent their own during
a reflective process?
Questions raised by Q6C
 How do researchers know when to stop
looking for sources?
 What do experienced researchers in a field
know about when to stop that newcomers
don’t know?
 How can newcomers to a field be
supported to do research with the savvy of
experienced researchers?
Computers & Writing 2009: Ubiquitous and Sustainable Computing
39
Q6C: Applied to history
Computers & Writing 2009: Ubiquitous and Sustainable Computing
40
Website analysis 1.0 beta
Computers & Writing 2009: Ubiquitous and Sustainable Computing
41
Computers & Writing 2009: Ubiquitous and Sustainable Computing
42
Website analysis 1.0
Computers & Writing 2009: Ubiquitous and Sustainable Computing
44
Spanish-American War 1.0
Spanish-American War 2.0
Spanish-American War 2.0
Q6C Lessons Learned
 Students already do low-level evaluation
 Web checklists don’t help much
 Thinking contextually with corroboration and
purpose in mind does
 Students can benefit from using Q6C
explicitly
Computers & Writing 2009: Ubiquitous and Sustainable Computing
50
Q6C for Students: Next Steps
 Design and re-design assignments with Q6C
 Use Q6C in course design, implicitly/explicitly
 Monitor and analyze results
 Tweak
 Repeat
 Infect others with the Q6C madness
Computers & Writing 2009: Ubiquitous and Sustainable Computing
51
Ongoing Work
 Continued classroom applications
 Modern American Civilization From 1877 (Tim)
 The rhetoric of Writing in the Workplace (Sarah)
 Further research and development
 Refinement of the Q6C working model
 Connect to literature on cognitive apprenticeship and
metacognition (e.g., Scardamalia & Bereiter (1983))
 Dissemination and partnership building
 Are you interested in using Q6C in your teaching? Talk
to us.
Computers & Writing 2009: Ubiquitous and Sustainable Computing
52
Thank You!
We wish to acknowledge the following:
 Center for Instructional Development and Research
 Practical Pedagogy
 Our students
 Suzzallo Espresso
 The Internet
 You
QUESTIONS?
Computers & Writing 2009: Ubiquitous and Sustainable Computing
53
Extra Slides
 Descriptions of each Q6C Component
 Suggestions for Teaching With Q6C
 Suggestions for Students Using Q6C
Computers & Writing 2009: Ubiquitous and Sustainable Computing
54
Q6C: Question
 Maintain a skeptical frame of mind
 Ask questions relevant to your research
Computers & Writing 2009: Ubiquitous and Sustainable Computing
55
Q6C: Categorize
 In the context of your research, is this a
primary, secondary, or tertiary source?
 What type of site is it (website, blog, wiki,
database, etc.)?
Computers & Writing 2009: Ubiquitous and Sustainable Computing
56
Q6C: Critique Rhetorically
 What do the authors’ choice of words, tone,
font, display format, images, genre, and
argumentative strategies tell you about the
intended audience and the credibility and
reliability of this site? (‘Read’ the site.)
Computers & Writing 2009: Ubiquitous and Sustainable Computing
57
Q6C: Characterize Authorship
 Identify who created the content, when
they created it, and for what purpose.
 Single or multiple authors? Committee?
Institution? Critic? Expert? Unknown? Other?
Computers & Writing 2009: Ubiquitous and Sustainable Computing
58
Q6C: Contextualize
 Place the information collected in
conversation with your existing experience
and body of knowledge.
 Does it fit? How?
Computers & Writing 2009: Ubiquitous and Sustainable Computing
59
Q6C: Corroborate
 Assess how the content compares to other
sources.
 Is the content consistent, complementary, or
contradictory?
Computers & Writing 2009: Ubiquitous and Sustainable Computing
60
Q6C: Conclude
 Is the source credible?
 Is the source useful for your research goals?
 If not, find a new source, repeat Q6C.
Computers & Writing 2009: Ubiquitous and Sustainable Computing
61
Suggestions for Teaching With Q6C
Remember that subject-area experts automatically
perform the Q6C process, whereas novices need to
consciously perform each step when learning how to
assess a source's credibility and usefulness.
 Identify which components of Q6C you want to
emphasize and scaffold the assignment’s research
process so that students learn new skills incrementally.
 Teach that research is about a process, not about a
product.
 Construct assignments that engage your students in
authentic research practices for your target discipline.
Computers & Writing 2009: Ubiquitous and Sustainable Computing
62
Suggestions for Teaching With Q6C
Tips for designing assignments and activities:
 Make explicit the outcomes of the assignment and
encourage reflection to help move students to the
meta-cognitive level.
 Consider choosing research topics that you are not
an expert in so that you can share the discovery
process with the students.
 Perform the assignment yourself or with a colleague
prior to class in order to anticipate student
responses.
Computers & Writing 2009: Ubiquitous and Sustainable Computing
63
Suggestions for Teaching With Q6C
Tips for designing assignments and activities:
 Make explicit the outcomes of the assignment and
encourage reflection to help move students to the
meta-cognitive level.
 Consider choosing research topics that you are not
an expert in so that you can share the discovery
process with the students.
 Perform the assignment yourself or with a colleague
prior to class in order to anticipate student
responses.
Computers & Writing 2009: Ubiquitous and Sustainable Computing
64
Suggestions for Students Using Q6C
 Q6C is a heuristic and not a checklist. You do not
need to answer every question for every source.
Some sources will require only asking one
component of Q6C; other sources will require
repeated application of Q6C.
 Remember that there are two questions you should
ask for every source: Is it credible? Is it useful?
Answers to these questions are not absolute: a
source can still be useful but still of dubious reliability.
Computers & Writing 2009: Ubiquitous and Sustainable Computing
65
Suggestions for Students Using Q6C
 When you think you have found a credible source,
perform one last check by applying Q6C a little
further. A review of Characterize Authorship,
Contextualize, and Corroborate can reveal
important, overlooked details.
 Practice using Q6C in other classes and daily
readings. As you gain experience, you will begin to
automatically use Q6C to evaluate all kinds of
information.
 Remember: Research is a process, not a product.
Computers & Writing 2009: Ubiquitous and Sustainable Computing
66