Practical Strategies for Student
Metacognition
Technology, Innovations, & Pedagogy Conference
CSU Fresno
August 18, 2014
Ed Nuhfer – Director of Faculty Development, Director of Educational
Assessment and Professor of Geology (University of Wisconsin, University of
Colorado, Idaho State University, California State University (retired)
enuhfer@earthlink.net Phone 208-241-5029
“High level teaching" is "…teachers
teaching the students something,
instructing them in how to produce
something, and giving them
instruction as to the processes of
learning.”
Hattie, J. (2009). Visible Learning: A Synthesis of over 800 MetaAnalyses Relating to Achievement. New York, New York:
Routledge. P. 11.
Metacurriculum for Metacognition
Instruments
Knowledge or Skill
Learning-across-curriculum
modules
Strategies and developing a
learning philosophy
Reading Reflections
Reflection & Monitoring
Reflection, Monitoring &
Evaluation
Goal-setting, Monitoring &
Evaluation Developing selfassessment & self-efficacy
Exam Wrappers
Knowledge Surveys
Learning Journals
Evaluation & Goal-setting
Developing self-assessment &
self-efficacy
Metadisciplinary Concept
Inventories (Assessment)
Reasoning, respect for diverse
ways of knowing, overarching
philosophy for becoming
educated
Available Learning-Across-the-Curriculum Modules
From CSU Faculty Developers
Used by permission of S. E. Petersen. Positron emission tomographic studies of the processing of
single words.
Tools and Scales of Learning Challenges
“Exam Wrappers”
95% for the exam plus 5% for submitting the following
metacognitive reflection:
1. What did I do well on and why?
2. What did I do less well and why?
3. When I encounter a similar challenge, what
strategy will I use to perform better?
Reflective Learning Journals (1)
1. Summarize your
assumptions, beliefs,
understanding that you
had when you first
encountered the topic
assigned here, before
you started to engage it
during learning it in this
course
1. Assessing what one
thinks at the start of a
task is a common theme
in metacognition. A free
writing exercise given at
the introduction of a
new topic is ideal for
helping students capture
their baseline
knowledge and archive
that for later use in a
journal entry.
Reflective Learning Journals (2)
2. Disclose new
vocabulary that you
acquired during your
study of this topic.
Include vocabulary
that you may have
acquired as part of
your process in items
4-6 below.
2. Building vocabulary
increases reading
comprehension and
personal ability to
access information.
Reflective Learning Journals (3)
3.
Discover and describe
connections that you can make
between the topic and several
other class sessions, exercises,
readings, homework. Note in
SPECIFICS what you found from
making these connections that
informed or expanded your
understanding of the topic. Be
sure to note how this learning is
relevant to one or more of the
published learning outcomes for
the course. Refer to the specific
outcome(s).
3.
Metacognitive reflection
uncovers connections that are
otherwise difficult to access.
Practice in seeking connections
between lessons and larger
ideas is immensely valuable.
Reflective Learning Journals (4)
4. So as avoid depending
on materials given to
you in class, take
action on your own
behalf by looking to
other resources. To
provide evidence of
this learning, describe
IN SPECIFICS what you
learned and from what
source(s).
4. Assigning students to
go beyond what
authority (instructors)
offers them moves
beginning students to
higher-level stages of
thinking.
Metacognition trains
independent thinking.
Reflective Learning Journals (5)
5. “Close the loop” by
comparing what you
knew in #1 against
what you know now.
5. Seeing one's growth
affirmed builds selfefficacy.
Journal template furnished as “blueline helpsheet”
Journal template furnished as “blueline
helpsheet”
• Download it from
http://profcamp.tripod.com/rljhelpsheet.rtf.
Knowledge surveys
• Metacognitive self-assessments
– If I think I can do this, how well can I do this?
– Can I look at the items and start to distinguish
which emphasize knowledge, skills or reasoning?
• For construction, consult the tutorials at
http://elixr.merlot.org/assessmentevaluation/knowledgesurveys?noCache=191:1306864598
Knowledge surveys…
Our challenge is to keep students in
constant contact with their metacognition…
• KS ITEM – COURSE CONTENT: 0 1 2 I can explain why it
was necessary to first understand the principles of original
horizontality, superposition, and crosscutting relationships
before scientists could begin to derive the theory of plate
tectonics.
• KS ITEM — METADISCPLINARY OUTCOME: 0 1 2 I can
employ examples to illustrate key differences between the
ways of knowing of science and of technology.
• KS ITEM — STAGE IN DEVELOPMENT OF THINKING: 0 1 2
My friend tells me: "Because everyone has a right to his or
her opinion, we respect others by accepting all opinions as
equally valid.” I can recognize the stage of intellectual
development revealed by that statement.
Hattie (2009, p 39 - 43) found present understanding of level of achievement as the most
significant attribute of students for meeting their future goals.
Self-Efficacy
• “If I have the belief that I can do it, I shall surely
acquire the capacity to do it, even if I may not
have it at the beginning.”
Mahatma Gandhi
KSs as an aid to teaching self-efficacy
• Self-efficacy—realistic belief in self regarding the
proficiency that one can attain with instruction and
support.
• Alfred Bandura established strong self-efficacy as
one of the most important predictors of student
success (www.des.emory.edu/mfp/self-efficacy.htm)
• PRE-COURSE: “Good heavens! Will we be able to
learn ALL this?"
• POST COURSE: "I really did learn that much.”
Correlations between performance
and several metacognitive selfassessments
(n = 1011)
First
Impression
Global Selfassessment
Knowledge
Survey itemby-item SelfAssessment
PostKnowledge
Survey
Overall SelfAssessment
Post-SLCI
Overall Selfassessment
(*n=590)
SLCI
Actual
Score
r = 0.29
r = 0.58
r = 0.45
r = 0.62*
Average
72 %
Average
75%
Average
79%
Average
76%
Average*
76%
Metadisciplinary Ways of Knowing
• You have a handout with outcomes for six
metadisciplines
• If not, grab it from
http://profcamp.tripod.com/metadoutcomeslist.pdf
Example of the reasoning component
of “Science literacy”
• Insert your own metadiscipline’s outcomes in
place of “science” as you consider what
follows.
• If a citizen understands science’s way of
knowing, how can we recognize this? How can
students recognize when they are “getting it?”
Concepts for Citizen Literacy in the Metadiscipline of Science
1. Science explains physical phenomena based upon testable information about the
physical world.
2. In modern life, science literacy is important to both personal and collective
decisions that involve science content and reasoning.
3. Doubt plays necessary roles in advancing science.
4. Scientists use evidence-based reasoning to select which among several
competing working hypotheses best explains a physical phenomenon.
5. A theory in science is a unifying explanation for observations that result from
testing several hypotheses.
6. Peer review generally leads to better understanding of physical phenomena than
can the unquestioned conclusions of involved investigators.
7. Science can test certain kinds of hypotheses through controlled experiments.
8. All science rests on fundamental assumptions about the physical world.
9. Science differs from technology.
10. Scientific knowledge is discovered, and some discoveries require an important
history.
11. Science employs modeling as a method for understanding the physical world.
12. Scientific knowledge imparts power that must be used ethically.
Concepts restated as 12 Outcomes for Science Literacy
Students will be able to…
1. Define the domain of science and determine whether a statement constitutes a
hypothesis that can be resolved within that domain.
2. Describe through example how science literacy is important in everyday life to an
educated person.
3. Explain why the attribute of doubt has value in science.
4. Explain how scientists select which among several competing working hypotheses best
explains a physical phenomenon.
5. Explain how "theory" as used and understood in science differs from "theory "as
commonly used and understood by the general public.
6. Explain why peer review generally improves our quality of knowing within science.
7. Explain how science employs the method of reproducible experiments to understand
and explain the physical world.
8. Articulate how science’s way of knowing rests on some assumptions.
9. Distinguish between science and technology by examples of how these are different
frameworks of reasoning.
10. Cite a single major theory from one of the science disciplines and explain its historical
development.
11. Explain and provide an example of how modeling is used in science.
12. Explain why ethical decision-making becomes increasingly important to a society as it
becomes increasingly advanced in science.
OK…Can we assess this stuff?
The Instrument
• For each outcome
– Construct several concept inventory items.
– Use established methods for drafting items that have been
developed in other concept inventories.
• In addition…
– Test reasoning, not factual knowledge.
– Administer the inventory under the conditions in which a
citizen will use common information.
…initial instrument constructed 2008-2010 through
the collegial efforts of
•
Edward Nuhfer, Faculty Development & Geology, Channel Islands
•
Jerry Clifford, Physics, Channel Islands
•
Christopher Cogan, Environmental Sciences & Resource Management,
Channel Islands
•
Anya Goodman, Biochemistry, San Luis Obispo
•
Carl Kloock, Biology, Bakersfield
•
Beth Stoeckly, Physics, Channel Islands
•
Christopher Wheeler, Geology, Channel Islands
•
Gregory Wood, Physics, Channel Islands
•
Natalie Zayas, Science Education & Environmental Sciences, Monterey Bay
Science Literacy Concept Inventory
• Incorporates 25 validated items that map to
the twelve concepts
• Reliability of .85
• Tested on over 16,000 students in about 30
institutions
Outcome. Student can define the domain of science and
determine whether a statement constitutes a hypothesis that
can be resolved within that domain.
Concept
• Science explains
physical phenomena
based upon testable
information about
the physical world.
Some Misconceptions
• Science is on a mission to
refute religion; scientists
study the paranormal;
untestable statements are
like scientific hypotheses.
Which of the following statements presents a hypothesis that science
can now easily resolve?
A. Warts can be cured by holding quartz crystals on them daily for a week.
B. A classmate sitting in the room can see the auras of other students.
C. Radio City Music Hall in New York is haunted by several spirits.
D. People with chronic illnesses have them as punishment for past misdeeds.
Which of the following statements presents a hypothesis that science
can now easily resolve?
A. Warts can be cured by holding quartz crystals on them daily for a week.
B. A classmate sitting in the room can see the auras of other students.
C. Radio City Music Hall in New York is haunted by several spirits.
D. People with chronic illnesses have them as punishment for past misdeeds.
What did we learn that made this so
interesting?
• Starting with…Do experts (professors)
outscore novices (students)?
YES! They do! Also, students on average do come to us with some
science literacy: zero literacy = about 25% (random guessing).
Does knowing “stuff” advantage a person?
Professors in Metadisciplines by
Mean Score %
We can learn about our students’
science literacy
Our current GE science courses don’t produce
science literacy (N = 16,623)…
Development seems more apparent long
term through ranks (N = 16,599)…
The major gains are breaks that may result from attrition as well as growth in ability to reason.
We can learn much about our
students
First Generation Students: 65% vs 70%
(N = 16,203)
Students from families privileged with higher education generally achieve higher.
English as a First Language: 62% vs
70% (N = 16,182)
Students whose first language is English generally achieve significantly higher scores.
Commitment to Interest in Science:
66% vs 72% (N = 16,190)
Differences are already significant in freshman year, and becomes increasingly so with each rank.
Gender-equality (n = 16,617)
And this holds true so far in our study within every ethnic group except one.
Middle Eastern (n = 373)
Ethnicities Show Some Profound
Differences
They Differ Across Three Factors
Regression Equation to Account for
Socio-economic
advantages/disadvantages based on
16,263 undergraduates
• Corrected Score = 0.62424 - 0.032575 (First
Generation) +0.06206 (English as Native
Language) + 0.05323 (Science Interest).
Ethnicity adjusted for Socio-economic
advantages
SLCI by Institutions with Other
Measures
SAT
Verbal
1.00
SAT
Math
0.94
ACT
Composite
0.97
SLCIFrsh
0.87
SLCISoph
0.86
SLCIJr
0.85
SLCISr
0.87
SAT Math
ACT
Composite
SLCI-Frsh
0.94
1.00
0.98
0.86
0.82
0.77
0.78
0.97
0.87
0.98
0.86
1.00
0.88
0.88
1.00
0.84
0.94
0.81
0.95
0.84
0.87
SLCI-Soph
SLCI-Jr
0.86
0.85
0.82
0.77
0.84
0.81
0.94
0.95
1.00
0.96
0.96
1.00
0.95
0.93
SLCI-Sr
0.87
0.78
0.84
0.87
0.95
0.93
1.00
SAT Verbal
As Eric Gaze and the Quantitative Literacy test researchers have noted, these little
literacy tests may be powerful predictors of student success.
Institutions
Given that our GE science courses don’t produce
much increase in reasoning of science literacy…
For continued learning about metacognition, we recommend that
participants enroll with the Improve with Metacognition forum at
http://www.improvewithmetacognition.com/2014/07/