Alternative Assessment
with Electronic Concept
Mapping Software
Dianne Raubenheimer and Kevin Oliver,
NCSU
1. What is Concept Mapping
What is concept mapping?
• concept mapping is a technique for organizing and
representing information
• a true concept map must include:
– core concepts - usually enclosed in circles or
boxes
– relationships illustrated by lines and arrows
connecting concepts AND by propositions or
statements on those lines that explain the nature
of the relationship
Books
Pages
are made of
Binding
From: http://cmap.ihmc.us/
How are maps used?
• for assessment to gain insight into student
knowledge structures--valid
understandings and misconceptions
• for group thinking to collectively brainstorm
and structure issues relevant to a topic
• for planning to organize information for a
report, presentation, debate, or discussion
• for research to analyze how groups
respond to or perceive something
Concept Map Forms
• a concept map can take many forms,
provided it includes concepts and
illustrates relationships
through BOTH lines AND
proposition statements:
• spider, with information
organized outward
around a central topic
Concept Map Forms
• hierarchy, with
information structured
from most to least
important; or from
most inclusive
and general to
less inclusive
and specific
Concept Map Forms
• flowchart, with
information ordered
in a linear format
Concept Map Forms
• systems, with
information ordered
like a flowchart
with inputs
and outputs
Non-Examples
A concept map is a
very specific thing; it
is NOT any of these
things:
A flow chart of a
process that has
no proposition
statements
explaining the
relationship
between stages
Non-Examples
an organizational chart
that has no
propositions
or statements
that explain
how different
people support
one other
Non-Examples
Story Webs: analyze various literary elements
(plot, characterization, theme), dissect characters,
plots and sequence of events, to learn about
composition
Non-Examples
• a mind map is similar to a concept map,
with a central topic
• main themes are
noted on "branches"
• main themes are
expanded into details
• propositions explaining
relationships between
concepts are not included
2. What is the Process for
Developing a Map?
Select Domain
• select a domain of knowledge somewhat
familiar to the person constructing the map
(e.g., air pollution)
• the domain should be
limited to a manageable size:
– a segment of text
– a lab activity
– part of a course
Identify Key Concepts in Domain
• list key concepts in domain (e.g. flowering
plants, dicotyledon, monocotyledon, etc…)
• order concepts from the most inclusive or
general to the less inclusive and specific
– Types of - dicotyledon, monocotyledon
– Roots – fibrous, tap, tuberous, aerial, etc.
– Leaves – simple, compound, pinnate, etc.
– Stems – herbaceous, woody
– Flowers – actinomorphic, zygomorphic
Construct Preliminary Map
• use software or post-it notes to arrange
concepts visually
• attempt different
structures:
hierarchical,
spider,
flowchart,
systems
Examine Relationships
• insert lines/arrows and write preliminary
proposition statements that explain
relationships between concepts
• look for cross-links between different
domains of knowledge identified (if topic
includes multiple domains)
• help students understand every concept
"could be" related to every other concept,
but they must evaluate and choose the
most important concept relationships
string maps
illustrate poor
understanding of
a knowledge
structure
3. What are Some Strategies
for Employing Concept Maps
in the Classroom?
Strategies: Complete Fill-in
• fill-in blanks in pre-structured map from a
list of concepts
• helps students
to consolidate
understanding
of concepts
and relations
among them
Strategies: Partial Fill-in
Strategies: Fill-in-the-Propositions
SOUND
General
Chuck Yeager
speed
v
longitudinal
wave
frequencies
wave length
Bell X-1
"Glamourous
Glennis"
October 17,
1947
frequency
f
20-20,000 Hz
15-50,000 Hz
1000-150,000 Hz
human
hearing
canine
hearing
bat hearing
Yeager's wife
Muroc Dry Lake
Beds, CA
Edwards AFB
Wave
Equation
Dr. Paul M. Rutherford
Instructor
Lee's Summit Technology
Academy
Lee's Summit, MO
Strategies: Fill-in-the-Propositions
SOUND
was first
broken by
General
Chuck Yeager
whose
is a
speed
v
travels in
longitudinal
wave
in a
frequencies
Bell X-1
"Glamourous
Glennis"
after
Yeager's wife
of
wave length
named
20-20,000 Hz
15-50,000 Hz
1000-150,000 Hz
the range of
the range of
the range of
human
hearing
canine
hearing
on
October 17,
1947
at
Muroc Dry Lake
Beds, CA
now known as
Edwards AFB
frequency
f
which is called
the
Wave
Equation
Dr. Paul M. Rutherford
Instructor
Lee's Summit Technology
Academy
Lee's Summit, MO
bat hearing
Strategies: Pre-Selected Term Mapping
Design a concept map to show your understanding of CO2,
a greenhouse gas, and its past and future effect on climate.
Use the following concepts:
•
•
•
•
•
•
•
•
•
•
fossil fuels
deforestation
clouds
atmospheric water vapor
ocean
CO2
Industrial Revolution
longwave solar radiation
global warming
human activity
• evaporation
• transpiration
• greenhouse gas
concentrations
• ice
• albedo
• photosynthesis
• feedback loops
• shortwave solar infrared
radiation
Strategies: Seeded Term Mapping
• also called "micro-mapping"
• provide students with a list of "starter" or
"seed" terms (5-6)
• students must use these terms in their
map AND use another 5-6 terms from their
own knowledge of the topic
Strategies: Collaborative Mapping
E.g. via Powerpoint and E-mail
• assign articles to read on a selected topic
• group students and provide each group with a
Powerpoint file to pass around via e-mail
• slide 1 contains the concept map; each student
adds 3-4 items per round (e.g., concepts, links,
propositions)
• slide 2 contains names of group members
• slides 3-4 contain comments from students
describing changes they made to the group file,
or relevant suggestions
Strategies: Advanced Organizer
• develop an expert map of a course or a
course topic
• provide to students as an advance
organizer prior to teaching and for review
purposes
Unguided Maps
• Once students are familiar with concept
mapping processes, they can also be
asked to do unguided maps, e.g.
– After reading a section of text,
– At the end of a section of course work.
– At the end of the course.
• This helps with identifying and tying
together core concepts, and also acts as a
learning tool for test revision.
4. How are Concept Maps
Used for Assessment?
•
‘Live’ links available at: http://www4.ncsu.edu/~cdrauben/concept%20maps/Assessing%20Concept%20Maps.html
Items Commonly Assessed
• were the correct concepts or nodes identified?
• do the lines and arrows correctly link concepts
together into clusters or hierarchies?
• do the propositions or statements on the lines
between concepts accurately describe how two
concepts are related?
• how detailed are the resources or elaborative
examples attached to the concepts (e.g., Web
links, articles, annotations, notes, excerpts cut
and pasted from a class discussion board, etc.)
4 Scoring Methods
•
•
•
•
(McClure, Sonak & Suen, 1999)
were the correct concepts or nodes identified?
do the lines and arrows correctly link concepts
together into clusters or hierarchies?
do the propositions or statements on the lines
between concepts accurately describe how
two concepts are related?
how detailed are the resources or elaborative
examples attached to the concepts (e.g., Web
links, articles, annotations, notes, excerpts cut
and pasted from a class discussion board,
etc.)
Structural Scoring
Novak Scoring Protocol
hierarchical
structure,
concept-links,
cross-links, and
examples
Sample Instruction Sheet (Dempsey et al, 2002)
1. Think about the following questions:
•
•
•
•
•
What is climate?
How is climate studied?
What factors determine climate?
How has climate differed in the past?
What can cause climate to change?
2. Recall the instructions for how to draw a
hierarchical concept map.
3. Construct a concept map of your ideas about
climate, starting here:
Climate
Sample Scoring Protocol
• a sample scoring protocol for this "climate" map
assessment:
– half a point for each of 12 specific concepts
– one point for each of up to four coherent, labeled,
hierarchical lines of connections related to prompting
questions
– extra points for good cross connections between
hierarchies
– note: large differences had to be reconciled between
two scorers; the two scores averaged
Relational Scoring
• Quality or importance of each individual
concept-link, without regard to the overall
structure of the map, e.g. one point for each
correct link and proposition.
• Modified relational scoring where, concept-links,
cross-links, and examples were assessed using
the following scale (West, et al) :
– invalid relationship between concepts (0 points);
– valid relationship between concepts but propositional label is
incorrect (1 point);
– valid relationship and propositional label correct but lacks
foundational or core relationship to subject matter (2 points); and
– valid relationship and propositional label and foundational or
core relationship apparent (3 points).
Holistic Scoring
• Using a rubric
• Example
http://www.markville.ss.yrdsb.edu.on.ca/
history/west/west_conceptmap.pdf
• Combining structural
and relational scoring
Other Scoring ideas
Concept List: Creating Instability
Cut these out to make it easier to move them around on the incomplete concept map
(Dempsey et al, 2002)
bottom of layer
warms
some parcels become warmer
than neighboring parcels
lapse rate
of some parcels
decreases
warm air
advects
into bottom
of layer
cold air
advects into
top of layer
cold air
advects into
top of layer
environmental
lapse rate
decreases
top of layer
cools more
than bottom
top of layer
cools
layer
undergoes
(large-scale)
forced lifting
lifted layer expands (deepens),
so top rises farther
and hence cools more than
bottom
water
evaporates
into parcels
Other Scoring ideas
Points for each
correct concept
placed in the
partially
completed map
Some issues
• structural scores increase significantly after
instruction (West, et al, 2002)
• relational method is the most reliable across
raters (Rye & Rubba, 2002, West et al, 2002)
• scores obtained from the correctness of
propositions (relational method) were most likely
to correlate with state and national standardized
tests (Rye & Rubba, 2002)
• but, relational method is less sensitive in
measuring changes and differences over time
(West, et a.)
5. What Electronic Tools
are Available to Support
Concept Mapping?
Mapping Tools
There are several commercial mapping tools:
Some Examples:
Microsoft Visio--business & technical diagrams
SmartDraw--mind map templates, timeline templates,
flowcharts, org charts, network diagrams, floor plans
Mapping Tools
• Kidspiration and Inspiration--used
heavily in K-12 settings, education
templates and clip art, move
between map and outline modes,
attach notes, audio, hyperlinks,
and other external resources,
export function
• MatchWare OpenMind--create
mind maps with templates, export
function
Mapping Tools
cMap– Free online tool built for collaborative
concept mapping, can attach video, images,
files, links, and other external resources, export
function
From Concept to Content Maps
• increasingly, mapping programs allow people to
attach resources to concepts (e.g., notes, links,
images, files), reminiscent of e-portfolio systems
• selecting and adding content resources to
concepts provides another layer of detail and
helps to show the types of information a student
associates with a concept (e.g., a specific
research article)
• encourage students to annotate or add notes to
explain the relationship of the attached
resources to the concept; otherwise you may be
forced to "guess" why they chose to include a
resource
References
• Dempsey, D., O'Sullivan, K., & White, L. (2002). Action research for
SFSU's NASA-NOVA course. Retrieved September 27, 2005, from
the World Wide Web:
http://funnel.sfsu.edu/courses/gm310/assessment/
• McClure, J. R., Sonak, B., & Suen, H. K. (1999). Concept map
assessment of classroom learning: Reliability, validity, and logistical
practicality. Journal of Research in Science Teaching, 36(4), 475492.
• Rye, J. A., & Rubba, P. A. (2002). Scoring concept maps: An expertbased scheme weighted for relationship. School Science and
Mathematics, 102(1), 33-44.
• Trochim, W.M.K. (2004). An introduction to concept mapping for
planning and evaluation. Retrieved September 27, 2005, from the
World Wide Web:
http://www.conceptsystems.com/papers/intro_article.cfm
• West, D.C., Park, J.K., Pomeroy, J.R., & Sandoval, J. (2002).
Concept mapping assessment in medical education: A comparison
of two scoring systems. Medical Education, 36, 820-826.