A Cognitive Apprenticeship for
Scientific Literacy Based on
Journalism
Joseph L. Polman, E. Wendy Saul, Alan Newman, Cathy
Farrar, Nancy Singer, Eric Turley, Laura Pearce,
Jennifer Hope, Glenda McCarty, and Cynthia Graville
University of Missouri-St. Louis
International Conference of the Learning Sciences 2010
This material is based upon work supported by the
National Science Foundation under Grant No. DRL-0822354
Acknowledgements
• NSF DRK-12 (Discovery Research K-12)
• Scijourn Project Team Members:
• Co-PIs: Joe Polman, Wendy Saul, Alan Newman,
Cathy Farrar
• UMSL: Mike Butler, Jennifer Hope, Laura Pearce,
Cindy Graville, Angela Kohnen, Glenda McCarty,
Nancy Singer, Eric Turley
• SLSC: Diane Miller, Terris Grimes, David
Hoffelmeyer, Rachel Mahan
• Advisory Board: Hubert Dyasi, Kevin Leander, Bill
Penuel, Carol Stearns
• External Evaluator: Brian Hand, Univ of Iowa
Scientific Literacy through Science
Journalism (SciJourn) Project
• Purpose:
promote public understanding of and
engagement with science and technology
• Means:
involve high school aged youth (14-18 years
old) in hybrid science journalism activities,
centered around the creation of an authentic
science news publication
Who, Where, and When
• Teachers working at schools
• Urban, suburban, and rural
• Biology, Chem, Physics, English, Journalism …
• Youth program run by SLSC
• Summer 20+ hrs/week for 5 weeks
• School year on Saturdays
• 2008-09: Alpha and beta semesters
• 2009-10: Pilot year
• 2010-11: Cadre I implements, Cadre II
comparison
Screen shot here
What does a science journalist do?
• Articulate an issue of contemporary interest
• Research it using multiple, credible sources
(print, interview, etc.)
• Digest and make sense of information, including
complex and technical information
• Construct an evidence-based story
• Put it out so that multiple people can understand
Relates to Scientific Literacy as …
• Public understanding of and engagement with
science
• Educate citizens to
• make informed decisions about STEM-related issues
within their personal lives
• engage critically in democratic discourse and debate
about STEM issues within the public sphere
• Requires
• Understanding and communicating the meaning and
significance of science and technology information
Toward an Articulation of Standards
for Scientific Literacy Based on
Journalism
Wendy Saul, Alan Newman, Nancy Singer,
Eric Turley, Laura Pearce, and Joe Polman
Standards in
original proposal
• To “flesh out” content standard in 6+1 so that
that scoring rubric would have more utility for
evaluating science text.
• To help teachers (and others) assess and respond
to SciJourn student writing
• To help us understand and document progress in
student work
These are “hybrid” standards
• What is realistic to expect from a HS student, not
a professional journalist?
• Promotes educators’ scaffolding based on
authentic practices
• We recognize the difference between what it
takes to get published in SciJourner as opposed
to what is useful for teaching (see SLAP -- no
stereotyping, lying, advertising or plagiarism)
The path
• Learning from professionals
• Led to focusing on certain aspects that seemed possible
to teach in a H.S.
• Led to first iteration of standards and
• Coding (Nancy and Eric) for more “tune up”
• while working in schools and at a youth development
program
• Led to Alan “seeing the same damn thing over and over
again”
• Other iterations
• Feedback from professionals
• Current version
The processes the standards need
to support
•
•
•
•
•
Articulate idea of current interest
Locate multiple, credible sources
Digest information
Shape an evidence-based story
Share in an authentic venue
Let’s look at the Standards
(It’s a recursive, iterative process we seek
to promote here)
Finding a topic:
Relevant
IV. Scientific information is relevant to readers.
IV.A: Reported findings are linked to local or
personal concerns and new applications are
considered.
IV.B: Readers’ implied questions are anticipated
and addressed.
Finding a topic:
Searching
I: Students are able to search effectively for and
recognize relevant, credible information sources,
especially on the Internet.
I.A: Knows how to use search engines and
search terms
I.B.: Privileges data from credible government
and nonprofit sites (e.g.; nih.gov and cancer.org)
and can ascertain the credibility of other
websites, using the “About Us” for clues.
I. C.: Keeps track of sources, including dates of
publication, author names and expertise and
home institution for purposes of attribution.
Locate multiple, credible sources
II.A Multiple sources
II.B Credible sources
II.B.1: Sources are relevant and reliable.
II.B.2: Appropriate stakeholders are consulted.
II.C.: Attributed sources
II.C.1: Except for accepted facts, ideas & theories, all
assertions, numbers, details and opinions are attributed.
II.C.2: The names of the experts/organizations are given
and their area of expertise/qualification is identified. Any
biases or potential conflicts of interest are noted.
II.C.3: Copyright rules are followed and relevant URLs
are given.
Digest information:
related to factual accuracy
V.A: The story structure indicates what is more and less
important from a reader’s and writer’s perspective. The
science connection is noted.
V.B: The article shows an understanding of the content and
is able to explain concepts and information, including
methods of scientific inquiry.
V.C: Precise language is employed and scientific terms are
used appropriately.
V.D: Quantitative measures are given in correct and
comparable units.
V.E: The latest/up-to-date information is presented.
V.F.: The headline and photo caption accurately reflect the
content of the story.
Shape the story
III. Scientific information, discoveries and
technologies are contextualized; broader
implications as well as reflections on past and
future understandings are noted.
III. A: The import of the information for society is
understood and sufficiently detailed
III.B: The article indicates which data/ideas are
widely accepted in the scientific community and
which are preliminary. The article sensibly
weights the import of findings and, where
appropriate, uses qualified rather than
declarative language.
Share
• Know your audience (back to relevance)
• Fore-fronting important information (back to
factually accurate)
• Attributing sources (back to multiple, credible)
• Responding to queries- authentic venue
Looking forward to seeing how well
they work as a frame for
• transfer tasks
• assessment of student texts and
• read alouds.
Rubrics
• In original grant rubrics were supposed to
enhance standards, i.e. be used primarily to
assess writing samples.
• We have come to understand the difficulty of
creating rubrics that are valid and reliable like
those developed by NWP and WestEd.
Typical rubric scoring system
designed by our teachers
Re: Attribution
All refs are
correctly
attributed
Most refs are
correctly
attributed
Some refs
are
correctly
attributed
Few refs are
attributed
No refs are
attributed
We are moving toward
specification sheets or
criteria assessments.
Attribution:
• Identify (underline and number) each ref needing
attribution.
• Did each ref include all info needed so that
someone could trace back the ref?
• The peer editor will:
check each ref. starring any they could not
locate.
Standards now serving as the hub.
• Curriculum Development: Useful for helping
students learning to read and write as well as
assess the quality of science news articles.
• Research: useful in helping identify skills we
associate with scientific literacy.
• Professional Development: useful for building
teacher knowledge
• Outreach: Utility beyond the news story and
actually beyond science, e.g. critical thinking,
reading in the disciplines
Closing thoughts:
• It appears that these standards have utility
beyond SciJourn (for reading as well as writing,
for teaching research skills and critical thinking.)
• Prods discussion on personal interests vs “big
ideas.” Provides a low bar for entering into
science discourse.
• Importance of strategies for helping teachers
respond to student writing-- something they and
we clearly identify as needing attention.
Designing Transfer Tasks to Assess
Scientific Literacy
Cathy Farrar, Joe Polman, Wendy Saul,
and Alan Newman
Purpose of this strand
Develop a series of tasks that provide insight into a
student’s ability to:
– Evaluate expertise
– Identify appropriate questions for experts
– Select and use multiple credible sources to gain more
information
– Determine coherence between text and image or
graphic
– Employ effective search strategies for more
information
– Link current science issues to self and others
Scientific Literacy
• Contextualization, content knowledge and skills
are generally agreed upon as components of
scientific literacy.
• Many of the current assessments focus on one or
two of these components
• Rarely is scientific literacy measured in its
“entirety” (Laugksch & Spargo, 1996) and few
composite measures that get at the complexity
have been published (Laugksch, 2000).
Specific Content
The moon has phases because...
a. It passes into the Earth's shadow every month
b. Only the side of the moon facing the sun is
illuminated
c. The moon causes tides in the oceans
d. None of the above
2009 Torstar Syndication Services. Permission granted for up to 5 copies. All rights
reserved.
Hot-button topic
Human Cloning
a. Human beings have been cloned.
b. While we are not yet able to clone humans other
mammals such as sheep, goats, rabbits, pigs, cats,
horses, and deer have been successfully cloned.
c. The only mammal that has been cloned is a sheep.
d. Cloning has only been successfully performed on
lower species such as insects.
http://www.vladozeman.com/Test.htm
Attitude or interest
Q1. How connected do you feel to the natural
world
• very connected
• somewhat connected
• not very connected
• not at all connected
http://science4grownups.com/archives/2009/04/10/general/california-academy-ofsciences-scientific-literacy-poll-part-1-322
Current Assessments
• Assessments such as these measure content
knowledge or attitude toward science without
much focus on contextualization or skills.
• The SciJourn Scientific Literacy Assessment
incorporates content knowledge, but does not
rely on it. Instead its focus is assessing skills
that students use to engage with science
information.
SciJourn Scientific Literacy Assessment
• Korpan, Bisanz, and Bisanz (1997) posited that
one hallmark of a scientifically literate
individual is “the ability to make effective
requests for information or to ask good questions
about scientific research”
• It is in this vein that initial transfer tasks were
developed
• Began as a series of tasks that utilize
informational text in the form of brochures, press
releases, or text with associated graphic or
image.
Development of the tasks
• A series of eight tasks were developed and
piloted in 10 different high school classrooms.
• Revision based on teacher feedback and pilot
student responses
• Alpha composite assessment created and piloted
• Reframing of the composite assessment using
Item Response Theory (Baker and Frank, 2001)
• Beta composite assessment (IRT) developed and
piloted in May/June 2010
Task Description
• Authentic examples of science texts they would
encounter
• Brochures
• Science news articles
• Science images and graphics with captions
SciJourn Scientific Literacy Assessment
Strand
Sample prompt
Locating credible
information
Which hyperlinks would you choose, explain why and
what you hope to find once you click this link?
Relevance
Why is this information important to the general
public?
Contextualization
Why might high blood pressure be more common in
teens today than 100 years ago?
Factual Accuracy
If you had to present this information to a community
group, what information would you check regarding
lead before including in the presentation and why?
Multiple Credible
Attributed Sources
What other information about SmartKid.com would
you like to have to determine its credibility?
Validity and Reliability
• Face and content validity— science educators
and science journalists
• Test-retest administration to non-study group
• Developing scoring framework
• Confirmatory factor analysis with pilot data and
SciJourn content standards
• Development of scoring guide
• Projected completion Fall 2010
Reframing and Assessing
Engagement with Science and
Technology
Jennifer Hope, Glenda McCarty,
and Joe Polman
This material is based upon work supported by the
National Science Foundation under Grant No. DRL-0822354
Purpose of this strand
• Inspired by Rosenzweig & Thelen’s Presence of
the Past
• Clarify the notion of engagement
• Align the idea with recent views
• Enhance understanding of ways youth are
engaged in science and tech in school and out
• Develop a valid means of assessing engagement,
and change in engagement over time
• For educators, as an indicator of engagement level
• As a program evaluation tool
Background
• Why engagement?
• What is engagement?
• time on task (Lee & Anderson, 1993; Singh, Granville, &
Dika, 2002)
•
•
•
•
interest, motivation (Thompson & Windschitl, 2002)
perceived ability (Markowitz, 2004)
interaction (Olitsky, 2007)
persistent (Azevedo, 2004)
• Where? In school, and out (Zimmerman & Bell,
2007)
Our Framework
• Interest, or openness to and stance toward the
science and technology in the moment
• Action, or actual involvement with science and
technology ideas and tools
• Identity, or ways that the science and technology
connects to people’s identity affiliations, in the
past, present, and future
(not understanding or knowledge per se)
Instrument Development Method
• Initial draft based on existing research (e.g. Falk,
et. al, 2001 & Durant, 1999) and our framework
• Interview-style administration
• Think-aloud with written administration
• Focus group
• Teacher & research team feedback
• Case study analysis
• Pilot administration (10 schools)
• Pre-survey
• Post-survey
Survey Design
Aspect
Number of
Questions
Types of Questions
Demographics 7
Gender, grade level, zip code
Interest
21
Level of interest in school, science, technology
Specific areas of interest in science/technology
Action
17
Frequency of activities related to science/tech
Ways of spending free time
Information sources utilized
Identity
19
Activities in everyday life related to science/tech
Views of self related to science/technology
Future plans
Case Studies
• Comparison of YEST survey results with
observation data from summer informal science
program participants
• Four cases were analyzed using themes of
interest, action, and identity
• Results of case study analysis utilized for further
modification of YEST instrument
Tre
High school junior; African-American male; 3rd year
participant in summer science program
Survey Responses
Rates science knowledge at
8.5/10 compared to peers
Science favorite subject, with
interests in robotics,
nanotechnology, and braincomputer interfaces
Regular consumer of science
and technology info
Wishes there were “more types
of courses” in science to take
Observational Data
Even more into sci-fi than science
Technology such an interest that
access became a distraction to
program goals
Keeps up to date on science and
tech news using iPhone apps
Expressed disappointment at not
going deeper into discussing “the
science of everything”
Khadijah
College-bound h.s. senior; African-American female attending
public school in the city; 4th year program participant
Survey Responses
Sees science as related to her
routine daily activities, but
more interested in technology
Particular interests in
chemistry and astronomy
Usually interested in science in
school, and sometimes outside
of school
Observational Data
Often “plugged in” to multiple
tech outlets at once
Thorough investigation of tech
tools used at tv station
Stated interests never surfaced
in projects
Selected personal topics, and
then sought science connection
Max
High school senior; First generation Eastern European
immigrant; 3rd year program participant
Survey Responses
Seeks science & technology
“for fun and personal
enjoyment” and “to learn more
about the topics”
Considers himself an excellent
learner with average grades
Notes that he will need to use
computers every day of his life,
including career interests
Observational Data
Despite early resistance to
writing, produced three science
stories in 5 weeks
Motivated to write about topics
of personal interest: the
SciJourn project, electric cars,
and tattoos
Associated himself with
project staff & participants
sharing technological interests
Alisha
African American high school senior; College prep and EMT
courses; 4th year program participant
Survey Responses
Observational Data
Good student with good grades Actions and identity which
aligns with science, but her
primary science interest was
Interactions with science and
instrumental
technology are mostly
“required” and/or “assigned”
Highly structured student,
excels in a traditional school
system model
In summer did not take
ownership of her writing
Seldom shared her scientific
knowledge with others
Summary
• Many challenges of only using one instrument
to assess a complex construct.
• Compared the youth's daily interactions with
science and technology and evaluated that
against their responses to the YEST survey.
• Analysis is to support the development and
modifications of the YEST survey and show that
it is possible to use only one instrument.
• Next steps: case studies and surveys of schoolbased participants
Reliability & Validity
• Face and content validity—teens, teachers, and
SciJourn research team
• Standardized codebook
• Test-retest administration to non-study group
• Still to come…
• Factor analysis with pilot data
• Development of scoring guide
Building an Apprenticeship
Community of Practice
for Science Journalism
Joe Polman, Wendy Saul, Alan Newman,
Laura Pearce, and Cindy Graville
The role of SciJourner
within the community
At least 362 articles drafted and submitted for
consideration
51 articles edited, revised, and published at the
online SciJourner
- 45 from students/teens
- 6 from teachers
- 36 traditional articles, 3 videos,
2 audio podcasts, 3 extended photo captions,
7 book reviews
The Role of SciJourner (continued)
Social networking aspect of online publication
53 comments on stories
>1700 hits on “Heartbreak of Potter’s
Syndrome”
>100 hits on 38 articles
5 Print editions of SciJourner
Hundreds of copies distributed
Excitement fueling student submissions, teacher
recruitment, and … an authentic writing task
driving scientific literacy
Role of Teachers in the Community
• Writers themselves
• Developed and critiqued lesson plans
• elaborated and adapted ideas
• created new models: e.g., info graphics
• Involvement in the research (feedback on
instruments, collection of artifacts, participation
in case studies, teacher action research studies,
etc.)
Boundary Objects and Brokering
• Connections across boundaries of communities
E.g., from journalism to school to ISE
• Boundary Objects: Material and conceptual
artifacts around which communities organize
practices
E.g., SciJourner, “credibility”
• Brokering: Agents from one community
negotiating practices in another community
E.g.,: journalists, scientists, education
researchers, teachers, students
Key boundary objects and
intersections of practice
1) The genre of the science news articles vs. the
traditional school-based "5-paragraph essay”
2) Story pitches that connect personal interest and
concern with relevant science
3) Internet searches for information, and uses of
Wikipedia
4) “Credibility” and “attribution” of sources
5) “Editing” for publication, vs. “responding” to
students to support their learning
6) Revisions in science classrooms
Genre wars: Science news article
vs. 5 paragraph essay
• Science news article
• Inverted triangle structure
• Created for communicative purposes, with pragmatic
constraints
• 5 paragraph essay
• Defines the norm of non-fiction writing in schools
• Created for assessment purposes
• Persuasive and expository essays
• Ask students to “set aside” in this class, but
allow for hybridity while meeting the standards
Pitching and crafting story ideas
• Personal connections and interests
• “Thinking about a tattoo?” (Health risks)
• “"Exceptions" to the Foster Care System” (HS
graduation rates)
• “Hunting Proves both Profitable and Crucial”
• “The Heartbreak of Potter's Syndrome”
• “Playing in Blood, Sweat, and Saliva” (Astroturf)
• “There’s science in everything”
• “Authentic” in the sense of personal meaning
Search
• Search engine
• Wikipedia – (paddling against a strong stream)
• For technical terms
• For sources
• Directed search
• WebMD, NIH, American Cancer Society …
• “There’s an organization for every issue.”
• At any website
• Read “About Us”
• Use other credibility clues
“Credible” and “Attributed”
• Terms have become a strong part of the
classroom discourse
• Negotiation and elaboration of their meanings
drives understanding
• Attribution issues
• Poor model of most familiar authoritative texts
(textbooks)
• “Works cited” from English class
“Editing” vs. “Responding”
Editing
Responding
Typical
agents/Brokers
Professional editor
Teacher
Purpose
Improve and polish
the article for
publication
Drive learning
Norms
Blunt, no-nonsense
Detailed
Begin with positive,
Remain supportive
(in practice, may be
broad & vacuous)
Tools
MS Word’s “Track
Changes”
Pencil or pen markup
“Designing for learning” while maintaining
authenticity of publication requires both
Revisions
• Writing in science classrooms (typically labs) is
seldom revised
• We are brokering a practice from
English/Communication Arts classrooms
(writing workshop models)
• Some resistance, but SciJourner as a boundary
object drives authentic purpose
Additional Information
http://www.scijourn.org
http://www.scijourner.org
Email polman@umsl.edu