Using concept map approaches to
communicate and present knowledge
University of Oulu, Finland
EDTECH A41857 (1 credit) – Challenges,
Problems, & Future of EdTech
Wednesday March 30, 2005
Dr. Roy Clariana
Penn State University
email: RClariana@psu.edu
home: www.personal.psu.edu/rbc4
"First we build the tools, then they build us!" -- Marshall McLuhan
1
goals

Your take aways:
• Some experiences with collaborative concept
•
•

mapping, mindmapping
Some understanding of how/why it works
Some examples that you could implement on
Monday morning in your classroom or in you
research
Your Digital Portfolio for future
reference and for sharing
2
1 credit option
Digital portfolio – Formulate as a group
a digital portfolio of mindmapping,
which you may utilize in the future in
your studies or work.
• For teachers, provide specific examples for
•
using mind mapping in your classroom
For researchers, provide specific examples
for using mind mapping in your research
3
2 credit option?


Digital Portfolio plus a
White paper – a 5-10 page (double-spaced)
persuasive review of some aspect of mind
mapping, i.e., scripting MM in CSCL, MM as
an artifact, etc. [Based on your intuition,
describe how a MM can work, this is your
first iteration of a “solution”. The White
papers is a “soft sell” for your “solution” that
describes the problem (90% of the
document) and then states clearly how your
solution solves the problem (10%). Avoid
straw man arguments.]
4
If you are interested…

Manuscript for presentation – I hope
that we can publish this experience, i.e.,
based on several projects we will do,
together we formulate questions, collect
and analyze data, write… (this will likely
go beyond the workshop time frame and
is also more open-ended)
For example:
How does interaction develop/evolve in online collaborative mind
mapping?
What scripts can support online collaborative mind mapping?
5
Agenda for today








Welcome and introductions all around
Q&A
Brief overview of concept maps
Intro to Cmap tools software
Brainstorm activity (group roles)
Set up Project 1 (see handout)
Set up Project 2 (see handout)
Does anyone have any student essays that we
can use in Project 3 on Monday?
Click here for projects handout
6
Some foundation stuff
some terminology


Vygotsky



Concept map – diagrams indicating interrelationships
among concepts and representing conceptual
frameworks within a specific domain of knowledge
(vanBoxtel)
contrast
Concept map – a visual set of nodes and arcs (a network
representation) that embodies the relationships among
the set of concepts. Also called knowledge maps,
mindmaps, semantic maps (Turns, et al.).
Nodes – terms/complexes/concepts (usually nouns,
things, examples, ideas, categories, people, locations…)
Links (arcs) – lines connecting nodes, usually labeled
with a relationship term (usually verbs)
Propositions – node-link-node combinations, also called
“soup” (ketti) by IHMC
Turns, Atman, & Adams, 2000
7
Mindmaps vs. concept maps
Bahr (2004) using concept maps to teach English to German students
8
sama
työtapa
liian pitkään
vaihto
kyllin
usein
demot,
konkreettiset
esimerkit
tekeminen
pelkkä
kalvoshow
työtavat
vs.
pelkkä
kuunteleminen
oppilaiden erot
huomaa
Mindmap of “group”
knowledge
hitaat
nopeat
erot
tukiopetus.
lisätehtäviä
(Anni,
Anna,
Paula,
Esa,
ja Herkko),
konkretisoi !
apu
yllätä !
luokkakohtaiset
source
hallway
erot
opettajan
kikkoja is the second floor
oppettajan
vaikutusmahdollisuudet
muista
huumori !
haasta,
kysele !
oma tarina
elävöittää
kytke
oppilaan
arkeen !
liikuta
oppilas
ylös
penkistä
ikäluokka
vaikuttaa
vilkas luokka
hiljainen
luokka
ei
palautetta
opettajalle
erityisen
paljon
kikkoja
näennäinen
keskittyminen ?
9
Mindmaps vs. concept maps

My question is, do concept maps or do
mindmaps fit better with the Finnish
language?
10
Tools to support mapping





Yellow stickies!! Pencil and paper may
be best for your classroom
Software – PowerPoint is pretty good
Inspiration is good but expensive
CMAP tool is free, but your tech person
will have to agree to support it
At least 22 other tools are available,
some free some not
11
Other concept map automatic
scoring approaches






CMap tools (IHMC) that we will use today
C-TOOLS – Luckie (PI), University of Michigan NSF grant
available: http://ctools.msu.edu/ctools/index.html
TPL-KATS – University of Central Florida (e.g., Hoeft,
Jentsch, Harper, Evans, Bowers, & Salas, 1990). TPLKATS: concept map: a computerized knowledge
assessment tool. Computers in Human Behavior, 19 (6),
653-657.
SEMNET – http://www.semanticresearch.com/about/
CMAT – Arneson & Lagowski, University of Texas,
http://chemed.cm.utexas.edu
Plus 22 other non-scoring map tools, Inspiration,
Kidspiration
12
Some previous uses of mapping



Usually involve individuals working
alone, and involve text in some way
Some collaborative strategies have been
used
Lets look at a few…
13
Using a student mindmap to “capture” a
text (note taking)
Mindmap notes
Textbook
text
Text text
text text text
text text text
text text text
memo
text
text
text
Examples?
student
14
Using a student mindmap to “capture”
research on a topic
text
Mindmap notes
Text text text
text text tex
Text text
Text text
text text tex
text text
Text text
textt
text text
textt
text
memo
text
text
www
text
Examples?
video
video
student
15
Then using the mindmap to write an essay
Mindmap notes
essay
text
memo
Text text
text text text
text text text
text text text
text
text
text
Examples?
student
16
Using a researcher drawn mindmap to
“capture” an interview transcript
Interview 1
Interview 1
text
Text text
text text text
text text text
text text text
attribute
theory note
issue
memo
text
text
text
The capability
and experience
of the person
coding the text
is critical…
coder
17
Using a group drawn mindmap to
“capture” an interview
Interview 1
text
text
text
text
Qs
The capability
and experience
of the person
coding the text
is critical…
interviewer
18
Example of dyad collaboration
Note the attentional effects of the artifact
(not online) Mindmap artefact
Verbal discussion (taped)
Observations:
On task
Abstract talk
3-propositions/min
Question
Answer
Criticize
Conflict
Elaboration
Co-construction
text
Analyze the discussion
text
text
Blah blah
blah blah
Blah blah
text
Blah blah
blah blah
Blah blah
The incredible value of talk!
Hannah
Yergin
Problem: Sometimes unscientific
notions are ingrained
Inferred:
Active use of prior knowledge
Acknowledged problems
Look for meaningful relations
Negotiation
Shared objects play an
important role in negotiation
and co-construction
van Boxtel, van der Linden, Roelofs, & Erkens (2002)
19
Chiu et al. example of an online
collaboration
Mindmap artefact
Mindmap session lasted 80
minutes. 3 x 12 online groups,
communicate by chat, 745
messages were exchanged
(avg. of 62 per group).
text
text
text
Online chat
H: WE should …
J: Did you see…
Y: Yeah, but …
Etc.
Etc.
Jari
The ‘other 2
members used
chat to “advise”
text
creates
Hannah
(lead)
Only the lead could
alter the mindmap
Researchers
Analyzed the chat text
And the mindmap
Yergin
p.22, Chiu, Huang, & Chang (2000)
20
Project 1 and 2




We will experiment with two online collaboration
approaches
Project 1 is a synchronous concept map
collaboration using Cmap tools software
Project 2 is an asynchronous concept map
collaboration using PowerPoint software and email
But next, we will try brainstorming with Cmap tools to
become familiar with the tools and process before
setting up Project 1
Click here for projects handout
21
Mindmap activity…
First Mind map CSCL roles…




Starter: You work as a discussion moderator. Your assignment is to engage your
group members to the discussion by asking questions and commenting. And if the
wrapper makes small summaries during discussion you can utilize his or her work to
raise new questions. Active participation in the discussions is essential.
Wrapper: Your assignment is to sum up the discussion. If you think it is easier you
can summarize frequently and weave ideas together. For example, if five participants
of your group are having a discussion about collaborative and co-operative learning
you can summarize their main points during the discussion. An alternative way is to
sum up the discussions in the end of article-videoclip task (and the last course
assignment). Please overview your group's discussions and make a brief summary
of the main topics. Active participation in the discussions is essential.
Group member: Your assignment is to participate actively into discussions by asking
questions making comments and stating arguments. You are expected to be a critical
inquirer.
Evaluator (an optional role): You are required to evaluate your group's work during
the course. Please focus on the group interaction and group dynamics, for example
how the starters, wrappers and group members performed during the discussions
and last course assignment. The tutors inform you when to perform evaluations.
Notice that you are also a deputy starter and a deputy wrapper if the originally
named persons are not available. If you are called to work as a starter or wrapper
please see the instructions given above. The role of evaluators are used only if you
have not had a role of starter or wrapper during this course.
22
Cluster analysis
Brainstorming
Sorting
(corpus list)
(move like terms closer)
Merging & Pruning
(combine like terms,
delete or move unlike terms,
synthesize terms)
Build consensus!
Naming Clusters
(name the categories/themes)
and if necessary
Sorting Clusters
(move like clusters closer)
E-document (to save/print)
Naming broad themes
(name the cluster of clusters)
23
Brainstorm, then make the map







Open IHMC Cmap tools
Fill in personal information on first use (I’ll tell
you what to type in here)
Click Other Places
Open brainstorm file
Click collaborate icon
if necessary
Type in your first name
Collaborate
24
Now go back and Mindmap activity…
add Small Group Roles
Group Task Roles
Initiator-contributor. Proposes new ideas or approaches to group problem solving; may
suggest a different approach to procedure or organizing the problem-solving task
Information seeker. Asks for clarification of suggestions; also asks for facts or other
information that may help the group deal with the issues at hand
Opinion seeker. Asks for clarification of the values and opinions expressed by other
group members
Information giver. Provides facts, examples, statistics, and other evidence that pertains
to the problem the group is attempting to solve
Opinion giver. Offers beliefs or opinions about the ideas under discussion
Elaborator. Provides examples based on his or her experience or the experience of
others that help to show how an idea or suggestion would work if the group accepted
a particular course of action
Coordinator. Tries to clarify and note relationships among the ideas and suggestions that
have been provided by others
Etc..
25
Project 1 – Cmap tools
synchronous collaboration
Set day and time to
join online …….
(see the Project handout)
26
Project 1
IHMC Public Cmaps conv v2 on Jan 22 2004
27
Project 1
Oulu EDTECH Public
28
Project 2 – Overview of “Pass
the soup”
PowerPoint file
Email to
Email to
Email to
Email to
(see the Project handout)
29
Project 2 – “Pass the soup”
PowerPoint file
Slide 1 – mindmap is developed bit-by-bit here by
the group by adding only 3 to 5 elements and then
emailing it to the next person on the list
1.
2.
3.
4.
Bob – bob@oulu.fi
Mary – mary@oulu.fi
Tiina – tinna@oulu.fi
Etc.
Instructions: Add 3 or 4
components, pass to the
next person…
B: I decided to add blah
and blah because I
am interested in
artifacts
M: I deleted Bob’s blah
because it is stupid,
and then added blah
T: I linked blah and blah
etc…
Slide 2 – numbered list of names of group
members with email address, other instructions
Slide 3, 4, etc. – comments about changes that
you want to make, suggestions, etc.
30
How to use ALA-Reader
Monday, April 4, 2005
31
Agenda for today






Debrief “pass the soup” activity, and come up
with a better Finnish name for it
Q&A
Brief overview of my concept map
assessment research
ALA-Reader demo (English language
essays)
Set up Project 3 for Finnish (see handout)
How can we find Finnish essays for use in
Project 3?
32
Final map for Project 2: Team 1
Why don’t we read from computer screens concept map?
computer to
communicate, paper
to study
easy to underline and
write notes on paper
paper more
portable
computer to store,
paper to read
with paper,
easier to multi-task
several paper pages
simultanosly viewed and
compared
working options/possiblities
and requirements
easier to make
good-looking
slides and copies
about drafts with
computer
Click her to
See progression
Of this map
comp screen is
smaller than paper
appearance
Computer screens
Poor screen
resolution (96 dpi)
computers require
paper has better
contrast
manual dexterity
of child and adult
feelings/perceptions
computers require
constant updates
computers skills
paper has weight,
texture, and feel
Group: Tanja, Henna & Roy
familiarity
33
Final map for Project 2: Team 2
underline
reliable but heavy to travel with
reliable
luminosity
doesn’t need
any hardware
resolution
eyes gets
mixed up
archive
Paper /Hard copy
paper easy to
read for eyes
Click her to
See progression
of this map
headache
Why don’t we
read text from
computers?
shoulder problems
never seen one
personal
preference
e-book
electronic
documents
easy to use
light to carry/ travel
technical
problems
multimedia
copyright
own comments
possibility to store
ergonomy
screen
size
able to
write notes
decision to
print
e-text easy to
copy/paste
Group: Maria, Paivi & Roy
etext feels
ephemeral
amount
of text
can make paper
copies my own
1 page
or less
2-10
pages
different
versions
book feels
comfortable
connections
print to
paper
screen
34
Debriefing





What happened?
What worked?
What did not work?
What would you do differently next time?
If you like, write this up as a team for
your final paper.
35
My research interests



prototypes
Mind map assessment – automatic scoring
software tool called ALA-Mapper
http://www.personal.psu.edu/rbc4/ala.htm
Essay assessment – automatic scoring
software tool called ALA-Reader
http://www.personal.psu.edu/rbc4/score.htm
for Latent Semantic Analysis (LSA) see:
http://www.personal.psu.edu/rbc4/frame.htm
36
Novak

Novak says “Concept maps were first developed in our
research program in 1972 as a way to represent
changes in children’s understanding of science
concepts over the 12-year span of schooling. We were
using modified Piagetian clinical interviews to assess
changes in their knowledge over time, but we found the
interview transcripts were too difficult to analyze for
changes in specific aspects of the children’s
knowledge. Instead we prepared concept maps from
the interviews.”
From: http://wwwcsi.unian.it/educa/mappeconc/jdn_an2.html
37
First uses… to represent knowledge
in a visual format
The primary parts of the system are the heart,
blood
tissue
cells,
within
and the
vessels.
bodyThe
by approximately
human heart, a9pump,
pints of
is made
bloodof
The
through
cardiac
human
100,000
muscle
circulatory
miles system
of vessels
is a transportation
The
Cardiac
system.
primary
muscles
Nutrients
partshave
of and
thea system
unique
oxygenfeature
are
arethe
carried
heart,
of to living
forming
blood
tissue
connections
cells,
within
and the
vessels.
between
bodyThe
bytwo
approximately
human
adjacent
heart,
cardiac
a9pump,
pints of
cells.
is made
This
bloodallows
ofthrough
cardiac
the100,000
muscle miles
cells toofcontract
vessels
powerfully
The
Cardiac
primary
andmuscles
quickly
partshave
involuntarily
of thea system
unique feature
are the heart,
of
Theforming
brain
blood
isconnections
cells,
unableand
to vessels.
increase
between
The
ortwo
decrease
human
adjacent
heart,
the cardiac
a pump,
heart's
cells.
isbeating
made
This allows
of cardiac
the muscle cells to contract
powerfully
The heart
Cardiac
isand
comprised
muscles
quickly of
have
involuntarily
foura unique
chambers;
feature
two of
upper
Theforming
chambers
brain isconnections
unable
called to
atriums,
increase
between
andortwo
decrease
lower
adjacent
the cardiac
chambers
heart's
cells.beating
called
This ventricles
allows the muscle cells to contract
The blood
powerfully
The heart
flowsisthrough
and
comprised
quickly
the right
of
involuntarily
four
side
chambers;
to the lungs
two
where
upper
The
it picks
chambers
brainupisoxygen.
unable
called to
atriums,
The
increase
blood
and
then
ortwo
decrease
returns
lower the
to
thechambers
right.
heart's
Next,
beating
called
it flows
ventricles
into the left where it I xxxx
The blood
The heart
flowsisthrough
comprised
the right
of four
side
chambers;
to the lungs
two
where
upper
it picks
chambers
up oxygen.
called atriums,
The blood
and
then
tworeturns
lower to
thechambers
right. Next,
called
it flows
ventricles
into the left where it I xxxx
The blood flows through the right side to the lungs
where it picks up oxygen. The blood then returns to
the right. Next, it flows into the left where it I xxxx
Novak interview data
Was science content knowledge
right
ventricle
left
atrium
pulmonary
vein
pulmonary
artery
lungs
remove oxygenate
CO2
blood
Mind Map
38
Finnish research with
concept maps…








Mainly for knowledge representation for instructional use but also for
representing the structure of a curriculum and for group
communication
Pasi Eronen, Jussi Nuutinenn and Erkki Sutinen,
(http://www.cs.joensuu.fi/pages/avt/concept.htm), Joensuu (computer
science)
Mauri Ählberg, Helsinki (education) and Erkki Rautama (computer
science)
University of Art and Design, Helsinki
(http://www2.uiah.fi/~araike/papers/articles/CinemaSense_Collaborativ
e_Cinemastudies_DeafWay2002.htm) (see also: Future Learning
Environment 3)
Text graphs (Helsinki): http://www.cs.hut.fi/Research/TextGraph/
Kari Lehtonen, Helsinki Polytechnic, concept maps as a portfolio
component (http://cs.stadia.fi/~lehtonen/DPF/dpf-berlin-02muotoiltu.doc)
Also School astronomy and Vocational Training and Education
4th IEEE International Conference on Advanced Learning
Technologies
Joensuu, Finland, August 30 - September 1, 2004
39
Concept map for assessment: score
validity???


oxygenate pulmonary
CO2 lungs vein
artery
ventricle
blood
atrium
left
atrium
pulmonary
vein
lungs
remove oxygenate
CO2
blood
Concept maps contains
propositions
These propositions
scores are generally
considered to be valid
and reliable measures
of science content
knowledge organization
(Ruiz-Primo, Schultz,
Li, Shavelson, CREST
in California. . .).
40
e.g.,…



Rye and Rubba (2002) reported that traditional
concept map scores were related to California
Achievement total test scores (r = 0.73). (Note
that Crocker and Algina say that validation
coefficients rarely exceed r=0.50.)
Concept maps (cognitive maps, concept maps)
may be an appropriate approach for assessing
structural knowledge (Jonassen, Beissner, &
Yacci, 1993).
For example, concept maps have been used to
visualize the change from novice to expert.
41
Scoring Concept Maps

Traditionally, concept maps are scored by
teachers or trained raters using scoring rubrics
(e.g., Lomask’s rubric)


Although this marking approach is time
consuming and fairly subjective, map scores
usually correlate well with more traditional
measures of science content knowledge
(multiple choice, fill-in-the blank, and essays)
Complex scoring rubrics decrease the concept
map score reliability (so keep scoring simple)
42
Scoring Concept Maps
C3 describes our automatic system for scoring
concept maps:
collect –>convert –> compare
1.
2.
3.
Collect raw map data
Convert raw data into a mathematical network
representation
Compare the mathematical network
representation of two maps (e.g., student to
teacher, student to expert, student to student)
43
1. Collect raw data



What raw data can a computer “extract” from
a concept map?
Term counts – in open-ended maps, count
required terms included
Propositions – a link connecting two terms
and a link label
Associations – geometric distance between
pairs of terms. Small values indicate
stronger relationship.
44
(n2-n)/2 pair-wise comparisons
Link and distance data
Link Array
left atrium
right ventricle
to the
pulmonary vein
moves through
pulmonary artery
a
b
c
d
e
f
g
left atrium
lungs
oxygenate
pulmonary artery
pulmonary vein
deoxgenate
right ventricle
b
c
d
e
f
g
1
1
1
1
0
0
0
0
0
0
0
1
0
0
0
-
d
e
f
g
passes into
Distance Array
to the
lungs
deoxygenated
a
0
0
0
1
0
0
oxygenated
a
b
c
d
e
f
g
left atrium
lungs
oxygenate
pulmonary artery
pulmonary vein
deoxgenate
right ventricle
Most approaches use only link label
information, usually called “propositions”.
a
120
150
108
73
156
66
b
36
84
102
42
102
c
120 114 138 54 84 144 138 42 114 120
-
45
Link and distance


Link data (propositions) – are the common way
to compare/assess concept maps
Distance data – not common, based on James
Deese’s (1965) ideas on the structure of
association in language and thought, cardsorting task approaches (Vygotsky in Luria,
1979, Miller, 1969), Kintsch and Landauer’s
ideas on representing text structure, and
neural network methods (Elman, e.g., 1995)
46
Using our Finnish Mind Map
example


Borrowed from Anni, Anna, Paula, Esa,
ja Herkko
Found in the hallway on the second floor
See next slide
47
sama
työtapa
liian pitkään
vaihto
kyllin
usein
demot,
konkreettiset
esimerkit
tekeminen
pelkkä
kalvoshow
työtavat
huomaa
erot
nopeat
lisätehtäviä
konkretisoi !
luokkakohtaiset
erot
yllätä !
opettajan
oma tarina
elävöittää
pelkkä
kuunteleminen
oppilaiden erot
oppettajan
vaikutusmahdollisuudet
muista
huumori !
vs.
kikkoja
kytke
oppilaan
arkeen !
liikuta
oppilas
ylös
penkistä
haasta,
kysele !
ikäluokka
vaikuttaa
tukiopetus.
apu
vilkas luokka
hiljainen
luokka
ei
palautetta
opettajalle
hitaat
erityisen
paljon
kikkoja
näennäinen
keskittyminen ?
48
hiljainen luokka
huomaa erot
kikkoja
luokkakohtaiset erot
oppettajan vaikutus-mahdollisuudet
oppilaiden erot
työtavat
vilkas luokka
vältä
Link array
hiljainen luokka
huomaa erot
kikkoja
luokkakohtaiset erot
oppettajan vaikutus-mahdollisuudet
oppilaiden erot
työtavat
vilkas luokka
vältä
-0
0
1
0
0
0
0
0
-0
1
1
1
0
0
0
-0
1
0
0
0
0
-0
0
0
1
0
-0
1
0
1
-0
0
0
-0
0
-0
--
Distance array
hiljainen luokka
huomaa erot
kikkoja
luokkakohtaiset erot
oppettajan vaikutus-mahdollisuudet
oppilaiden erot
työtavat
vilkas luokka
vältä
-127
245
79
214
161
234
73
302
-199
52
122
91
111
117
207
Collect Mind Map raw data
-225 -100 164 -290 93 205 -175 164 76 166 -288 68 232 105 227 -114 252 88 282 122 320
--
9 main terms selected here (ALA-Mapper max=30)
49
Selecting terms


Selecting important terms (and their
synonyms) is a critical step (for example,
singular value decomposition in LSA
derives terms). We use an expert(s) to
determine terms.
Goldsmith, Johnson, and Acton (1991)
50
predictive validity of PFNets directly
relates to the number of terms used
0,80
predictive validity
0,70
0,60
0,50
0,40
0,30
So, perhaps the predictive validity of
Concept Maps (and essays) directly
relates to the number of terms used
0,20
0,10
0,00
0
10
20
30
Number of terms
Goldsmith, Johnson, and Acton (1991)
51
2. Convert raw data into scores



Currently, we use a data reduction and comparison
approach called Pathfinder network representation
(PFNet, Schanveldt, 1990). Our future research will
consider additional approaches, such as MDS and
data-mining. http://interlinkinc.net/Pathfinder.html
PFNets describe the least weighted path to connect the
terms
Scores are established by comparing the participant’s
PFNet to a referent (expert) PFNet, and calculating the
number of common links (the intersection)
Visual example 
52
Finnish example:
PFNet for distance data
hiljainen luokka
vilkas luokka
luokkakohtaiset erot
oppilaiden erot
huomaa erot
työtavat
oppettajan vaikutusmahdollisuudet
vältä
kikkoja
PFNet for distance data
53
Compare student to expert referent
hiljainen luokka
O
vilkas luokka
vilkas luokka
hiljainen luokka
luokkakohtaiset erot
oppilaiden erot
huomaa erot
luokkakohtaiset erot
oppilaiden erot
huomaa erot
6 of 8 common links
O
työtavat
oppettajan vaikutusmahdollisuudet
oppettajan vaikutusmahdollisuudet
työtavat
kikkoja
vältä
Expert Referent PFNet
vältä
kikkoja
Student PFNet
54
#1st
Poindexter and Clariana




Participants – 23 undergraduate students in
intro EdPsyc course (Penn State Erie)
Food rewards for participation
Setup – complete a demographic survey
and how to make a concept map lesson
Text based lesson interventions –
instructional text on the “heart” with either
proposition specific or relational lesson
approach
Poindexter, M. T., & Clariana, R. B. (in press). The influence of relational and proposition-specific processing on
structural knowledge and traditional learning outcomes. International Journal of Instructional Media, 33 (2), in
press. link to doc file
55
Treatments


Relational condition, participants were required
to “unscramble” sentences (following Einstein,
McDaniel, Bowers, & Stevens, 1984) in one
paragraph in each of the five sections or about
20% of the total text content
Proposition-specific condition (following
Hamilton, 1985), participants answered three
or four adjunct constructed response questions
(taken nearly verbatim from the text) provided
at the end of each of the five sections, for a
total of 17 questions covering about 20% of the
total text content (no feedback was provided).
56
Posttests

Concept map (use 26 terms provided)

Multiple-choice tests (Dwyer, 1976)
• Link-based common scores
• Distance-based common scores
• Identification (20)
• Terminology (20)
• Comprehension (20)
57
Means and sd
Treatments
Posttests
Map-link
COMP
Map-prop
7.3
14.1
(5.4)
(4.6)
control
ID
15.1
(4.4)
TERM
12.3
(4.6)
Map-dist
Map-assoc
9.0
(3.6)
propositionspecific
16.3
(5.6)
14.6
(5.7)
13.8
(3.7)
16.5
(8.3)
11.5
(3.4)
relational
17.0
(2.6)
12.7
(3.5)
12.4
(3.0)
13.9
(9.4)
10.7
(4.6)
58
Analysis



MANOVA (relational, proposition-specific, and
control) and five dependent variables including
ID, TERM, COMP, Map-prop, and Map-assoc.
COMP was significance, F = 5.25, MSe =
17.836, p = 0.015, none of the other
dependent variables were significance.
Follow-up Scheffé tests revealed that the
proposition-specific group’s COMP mean was
significantly greater than the control group’s
COMP mean (see previous Table).
59
Correlations
ID
TERM
COMP
Map-prop
Map-link
Map-distance
Map-assoc
ID
-0.71
0.50
0.56
0.45
TERM
COMP
Map-link
Prop
-0.74
0.77
0.69
-0.53
0.71
-0.73
All sig. at p<.05
Compare to Taricani
& Clariana
next 
60
Taricani and Clariana –
Replication of Poindexter and Clariana
Term
Comp
Link data
0.78
0.54
Distance data
0.48
0.61
Taricani, E. M. & Clariana, R. B. (in press). A technique for automatically scoring open-ended concept maps.
Educational Technology Research and Development, 53 (4), in press.
61
Compare these two . . .
Taricani & Clariana
Term
Comp
Link data
0.78
0.54
Distance data
0.48
0.61
Poindexter & Clariana
Term
Comp
Link data
0.77
0.53
Distance data
0.69
0.71
62
# 2nd
Clariana, Koul, & Salehi


Participants – A group of 24 practicing
teachers enrolled in CI 400
Lesson intervention – while researching
online, completed concept maps in pairs
(newsprint & yellow stickies) to describe
the structure and function of the heart
and then individually wrote essays on
this topic from their maps.
Clariana, R. B., Koul, R., & Salehi, R. (in press). The criterion related validity of a computer-based approach for
scoring concept maps. International Journal of Instructional Media, 33 (3), in press.
63
Posttests
Essays
 Multiple-raters using holistic rubric
 Computer-derived LSA Essay scores
(http://www.personal.psu.edu/rbc4/frame.htm)
Concept Maps
 Multiple-raters using Lomask’s rubric
 ALA-Mapper PFNet link and distance
agreement with an expert
64
Correlation matrix
Human
Map
Essay
LSA
Link data
Distance data
Map
1
0.49
0.31
0.36
0.60
Computer
Essay
LSA
Link
1
0.73
0.76
0.77
1
0.83
0.71
1
0.82
1
p < .05 shown in boldface type.
Many investigators have noted the close relationship between maps and essays.
65
Overview: Tools to score Essays



ETS – PEG (Project Essay Grade), e-rater,
Criterion and other products…
http://www.ets.org/research/erater.html
Walter Kintsch (and Landau) at CU-Boulder –
Latent semantic analysis, many uses, i.e.,
score online training for the Army http://lsa.colorado.edu/
Vantage Learning essay scoring products http://www.vantagelearning.com/
ALA-Reader: http://www.personal.psu.edu/rbc4/score.htm
66
ALA-Reader
Text
PFNet
… an electrical signal starts the
heartbeat, by causing the
atrium to contract. The blood
then flows through the
pulmonary valve into the
pulmonary artery and then into
the lungs. Once inside the
lungs, the blood gives up the
carbon dioxide (cleansed) and
receives oxygen. This
oxygenated blood …
Link array
atrium
contract
P valve
P artery
lungs
cleansed
oxygenated
67
# 3rd
Clariana & Koul


Participants – Again, a group of 24
practicing teachers enrolled in CI 400
Lesson – while researching the topic “the
structure and function of the heart”
online, students completed concept
maps using Inspiration software and
later wrote an essay on this topic from
their maps.
Clariana, R.B., & Koul, R. (2004). A computer-based approach for translating text into concept map-like
representations. In A.J.Canas, J.D.Novak, and F.M.Gonzales, Eds., Concept maps: theory, methodology,
technology, vol. 2, in the Proceedings of the First International Conference on Concept Mapping, Pamplona, Spain,
Sep 14-17, pp.131-134. http://cmc.ihmc.us/papers/cmc2004-045.pdf
68
Posttests
Essays
 Multiple-raters using holistic rubric
 Computer-derived LSA Essay scores
(http://www.personal.psu.edu/rbc4/frame.htm)
Concept Maps
 Multiple-raters using Lomask’s rubric
 ALA-Mapper PFNet link and distance
agreement with an expert
 ALA-Reader PFNet link scores (from 1 to 5)
(so far, only looked at essay scores) 
69
ALA-Rater PFNet scores


The scores for
each text and
rater-pair are
shown ordered
from best to
worst.
ALA-Reader
scores were
moderately
related to the
combined text
score, Pearson r
= 0.69, and
ranked 5th
overall.
70
Comments and Questions

??
71
Demo ALA-Reader





Download ALA-Reader.exe
Create terms file (can include 2 synonyms)
Create 2 expert baseline reference texts called
expert1.txt and expert2.txt (i.e., Instructor, best
student)
Use it (type in the students essay file name)
Files created
•
•
Summary file called report.txt
Multiple *.prx files (PRX folder)
Available at: www.personal.psu.edu/rbc4
72
Other methods for eliciting and
representing knowledge
structure
Monday, April 11, 2005
73
agenda
Today is a hands-on demonstration day
 Brief overview of the ideas
 SPSS for representing
 Pathfinder
 KU-Mapper
My intent, you will know enough to begin to use these approaches
74
Eliciting structural knowledge

oxygenate pulmonary
CO2 lungs vein
artery
ventricle
blood
atrium
left
atrium
pulmonary
vein
lungs
remove oxygenate
CO2
blood

Every method for
eliciting knowledge
should be viewed as
“sampling”
Caution, never forget
the likely effects of
contiguity (time, space,
etc.) dominating over
semantics (meaning)
75
Dave’s ideas
Knowledge
elicitation
Knowledge
representation
Jonassen, Beissner, & Yacci (1993), page 22
Knowledge
comparison
76
Dave’s ideas
word
associations
semantic
proximity
similarity
ratings
card
sort
Knowledge
elicitation
relatedness
coefficients
ordered
recall
graph
building
quantitative
graph
comparisons
free
recall
additive
trees
hierarchical
clustering
C of PFNets
qualitative
graph
comparisons
Knowledge
comparison
Knowledge
representation
Trees
scaling
solutions
expert/
novice
Dimensional
Networks
MDS – multidimensional scaling
ordered
trees
minimum
spanning
trees
link
weighted
Jonassen, Beissner, & Yacci (1993), page 22
Pathfinder
nets
principal
components
cluster
analysis
77
Eliciting structural knowledge




Vygotsky (in Luria, 1979); Miller (1969) cardsorting approaches
Deese’s (1965) ideas on the structure of
association in language and thought
Kintsch and Landauer’s ideas on
representing text structure, and latent
semantic analysis
Recent neural network representations (e.g.,
Elman, 1995)
78
Analyzing Deese free
association data with MDS

Hands-on with MDS in SPSS
• A good description of MDS:
•

http://www.statsoft.com/textbook/stmulsca.html
(Aside: a good description of Factor analysis:
http://www.statsoft.com/textbook/stfacan.html )
Hands-on with Pathfinder KNOT
79
bug
flower
yellow
fly
bird
wing
insect
moth
Deese, free recall data (p.56)
moth
100 12 12 12 11
1
0
4
insect
12 100 9
9
17
1
1
33
wing
12
9 100 44 19
0
0
3
are shown
bird
12100 participants
9
44 100
21 a 1list of 0
3
time, and
fly
11related
17 words,
19 one
21 at a100
1
1
8
asked to free recall a related term
yellow
1
1
0
1
1 100 7
0
flower
0
1
0
0
1
7 100 2
bug
4
33
3
3
8
0
2 100
cocoon
11 10
2
2
6
0
0
7
color
0
1
0
1
1
17
3
0
Full array (n * n): 19 x 19 = 361
blue
0
1
0
1
2
23
7
0
Half array ((n – n)/2): ((19 x 19) –19 )/2 = 171
bees
2 in language
3 and thought.
10 Baltimore,
10 MD: John
6 Hopkins
2 Press, page
2 56 805
Deese, J. (1965). The
structure of associations
2
moth
insect
wing
bird
fly
yellow
flower
bug
cocoon
color
blue
bees
summer
sunshine
garden
sky
nature
spring
butterfly
butterfly
spring
nature
sky
garden
sunshine
summer
bees
blue
color
cocoon
bug
flower
yellow
fly
bird
wing
insect
moth
Deese, free recall data (p.56)
100 12 12 12 11
1
0
4
11
0
0
2
2
5
1
1
1
1
15
12 100 9
9
17
1
1
33 10
1
1
3
0
0
0
0
1
0
12
12
9 100 44 19
0
0
3
2
0
0
10
0
0
0
0
3
0
13
12
9
44 100 21
1
0
3
2
1
1
10
0
1
0
1
5
0
12
11 17 19 21 100 1
1
8
6
1
2
6
0
3
0
2
4
0
11
1
1
0
1
1 100 7
0
0
17 23
2
2
7
5
2
4
3
5
0
1
0
0
1
7 100 2
0
3
7
2
1
6
18
2
6
2
6
4
33
3
3
8
0
2 100 7
0
0
5
0
0
0
0
2
0
4
11 10
2
2
6
0
0
7 100 0
0
4
1
1
1
0
2
0
22
0
1
0
1
1
17
3
0
0 100 32
0
0
2
0
8
0
0
0
0
1
0
1
2
23
7
0
0
32 100 1
2
4
4
46
3
2
2
2
3
10 10
6
2
2
5
4
0
1 100 1
2
3
0
4
2
7
2
0
0
0
0
2
1
0
1
0
2
1 100 5
2
0
1
10
0
5
0
0
1
3
7
6
0
1
2
4
2
5 100 2
3
2
15
4
1
0
0
0
0
5
18
0
1
0
4
3
2
2 100 0
4
4
2
1
0
0
1
2
2
2
0
0
8
46
0
0
3
0 100 0
1
0
1
1
3
5
4
4
6
2
2
0
3
4
1
2
4
0 100 2
3
1
0
0
0
0
3
2
0
0
0
2
2
10 15
4
1
2 100 2
15 12 13 12 11
5
6
4
22
0
2
7
0
4
2
0
3
2 100
Full array (n * n): 19 x 19 = 361
Half array ((n2 – n)/2): ((19 x 19) –19 )/2 = 171
Deese, J. (1965). The structure of associations in language and thought. Baltimore, MD: John Hopkins Press, page 56
81
Using MDS in SPSS






Start SPSS and open the deese.sav file
Under Analyze, select Scale, then select
Multidimensional Scaling (ALSCAL)…
Move Variable from left to right
Create distances from data
Model
Next page
Options
82
Select all of these
83
Multi-dimensional scaling (MDS)
of Deese data
Derived Stimulus Configuration
Euclidean distance model
1,5
spring
garden
1,0
summer
sunshine
nature
bees
Dimension 2
flower
cocoon
0,5
butterfl
0,0
moth
-0,5
yellow
blue
-1,0
fly
insect
color
sky
wing
bird
bug
-1,5
-2
-1
0
1
Dimension 1
84
Side issue, the MDS obtains alternate (e.g.,
enantiomorphic) visual representations
Helsinki
Oulu
Both are “correct”.
Is this map correct?
Tampere
Pori
Jyväsklyä
Jyväsklyä
Pori
Tampere
Helsinki
Oulu
geographic data, for example, may be oriented in different ways
85
Derived Stimulus Configuration
How good is the representation?

Euclidean distance model
many dimensions
(as many as 19)
reduced to 2
dimensions
Check the “stress”
value to estimate
how strained the
results are
1,5
spring
garden

Dimension 2
1,0
summer
sunshine
nature
bees
flower
cocoon
0,5
butterfl
0,0
moth
-0,5
yellow
blue
-1,0
fly
insect
color
sky
wing
bird
bug
-1,5
-2
-1
0
1
Dimension
An algorithmic, power, approach rather than based on a model
so 1
no assumptions about data structure are required…
86
Side trip

Wordnet: http://wordnet.princeton.edu/
http://wordnet.princeton.edu/cgi-bin/webwn

What is the Visual Thesaurus? – The Visual
Thesaurus offers stunning visual displays of
the English language. Looking up a word
creates an interactive visual map with your
word in the center of the display, connected to
related words and meanings.
Type “bird” in at:
http://www.visualthesaurus.com/trialover.jsp

87
Pathfinder Network (PFNet)
analysis



Pathfinder is a mathematical approach for representing
and comparing networks, see:
http://interlinkinc.net/index.html
Pathfinder data reduction is based on the least weighted
path between nodes (terms), so for example, Deese’s
171 data points become 18 data points. Only the salient
or important data is retained.
Pathfinder PFNet uses, for example:
•
•
•
Library reference analysis
Measuring Team Knowledge (Nancy J. Cooke)
next slide
Use google to see many more
88
Pathfinder for cognitive task
analysis
Shope, DeJoode, Cooke, and Pedersen (2004)
89
PFNet of same data
Now let’s try Pathfinder analysis of the
same Deese data set…
 Find the pfnet folder
 Double-click to run PCKNOT.bat (notice
the bat extension, see next slide below)
 We will do it together
90
Select the right PCKNOT file
91
PFNet of Deese data
sky
summer
blue
spring
sunshine
color
garden
yellow
flower
nature
butterfly
cocoon
moth
wing
bird
bees
fly
insect
bug
92
Derived Stimulus Configuration
MDS and PFNet of
Deese data
Euclidean distance model
1,5
sky
summer
spring
blue
color
garden
1,0
garden
sunshine
yellow
flower
butterfly
cocoon
wing
bird
fly
nature
bees
cocoon
nature
0,5
butterfl
0,0
moth
moth
-0,5
bees
yellow
blue
-1,0
fly
insect
color
sky
insect
bug
sunshine
flower
Dimension 2
spring
summer
wing
bird
bug
-1,5
Pathfinder KNOT PFNet
-2
-1
0
SPSS 1MDS
Dimension
1
93
MDS and PFNet data reduction


MDS uses all of the data points to
reduce the dimensions in the
representation, and so may be
improperly driven by noise in the data or
by unimportant data points
Pathfinder uses only the most important
data
94
Transition to your real life
example


Finally, you will collect *real* data (using
my KU-Mapper software) and
analyze it with Pathfinder KNOT
95
KU-Mapper



Your data, determine 15 important terms
in your research area (Finnish and
English), create a “terms.txt” file with the
15 terms
Run KU-Mapper (do all 3 tasks: pairwise, list-wise, and card sort)
Use KNOT to analyze and compare all
three prx files
Download KU-Mapper from: http://www.personal.psu.edu/rbc4/KUmapper.htm
96
Debrief your data activity





What happened?
What worked?
What did not work?
What would you do differently next time?
If you like as your final paper, describe
how you might use this approach.
97
Final thoughts…


I enjoyed working with you
If you want a credit,
• Email to let me know this
• Then be sure to send me you paper via email
as soon as possible
98
Stop here
99
Amount of collaboration
Possible research question on optimal scripts:
Under- vs. over-scripting CSCL
linear
S-curve
Amount of scripting 
Amount of scripting 
J-curve
Amount of scripting 
Some possibilities
100
generative learning strategies
+
++
Generative learning (Jonassen, 1988)
 recall - repetition, rehearsal, review, mnemonics
 integration - learner paraphrases, generates
questions, generates examples
 organization - learner analyzes key ideas by creating
headings, underlining keywords, outlining,
categorizing (i.e., invent table categories, populate a
table with existing ideas)
 elaboration - generate mental images, create
physical diagrams, sentence elaboration (i.e., invent
stuff to fill cells in a table)
101
I just "think" systemically and "ndimensionally" on paper, with imagery…
My essential skill is simply--If you can explain it to me, I can draw a picture of it.
It doesn't matter if it's something totally new to me, if you can make a coherent
explanation, and let me understand it. I can "visualize it" and make a picture
that shows you what you said.

This is why I work in aerospace. I'm able to sit down with SME's (Subject Matter
Experts-in any discipline), let them do a "data-dump" and put a sketch in their
hand at the end of the conversation that "say's it all". This skill is vital to helping
disparate technical types talk to each other (communication across cultural
barrier of the "dialect" of the various technical disciplines). It also provides a
way for ideas to get from that rough-semi coherent stage and into a practical
and "do-able" condition.
For example,

One day I found myself working a Kelly Temp job for a bunch of Boeing System
Analysts doing a JAD (joint application development) project to design a
computing architecture for a new tooling system for the 777. The first drawing
came by accident, started a huge argument, and eventually (2 weeks later)
resolved in a group wide "a-hah"... that put everyone on the same wavelengthallowing the new system to be built a lot more "right" than usual, quicker than
usual.

From: http://visual.wiki.taoriver.net/moin.cgi/MichaelErickson
102