A Comparison of the Effects on Test Performance of Computer-Based Concept Mapping to
Computer-Based Exercises Using Google Docs
Courtney Blackhurst
Penn State University
1
Abstract
This paper examines the use of concept mapping as a technology tool in a social studies
classroom to help students reach higher order thinking skills such as analysis and comparison.
Students were placed in two groups, listened to the same lecture on each of the six approaches to
psychology and were then asked to either fill in a worksheet with recall information or create a
concept map outlining their knowledge. The study provided evidence to suggest that there is no
difference between the use of recall worksheets and the creation of concept mapping on a
multiple choice and essay test, where test questions asked student to distinguish between the
approaches to psychology, apply them, and analyze their differences.
2
Table of Contents
I.
Introduction
a. The Problem………………………………………………………………………4
b. The Purpose of the Study…………………………………………………………4
c. The Research Question…………………………………………………………...5
II.
Review of Literature
a. Theoretical Justification……………………………………………………….....6
b. Observational Justification……………………………………………………….7
III.
Methods
a. Research Designs………………………………………………………………….8
b. The Participants…………………………………………………………………...8
c. The Materials……………………………………………………………………...9
d. Procedures………………………………………………………………………..11
IV.
Analysis and Results
a. Research Question……………………………………………………………….18
b. Descriptive Statistics…………………………………………………………….18
c. Statistical Analysis………………………………………………………………19
V.
Discussion……………………………………………………………………………21
VI.
Limitations of the Study……………………………………………………………...23
VII.
Implications and Conclusions………………………………………………………..23
VIII.
Appendix……………………………………………………………………………..25
3
I.
Introduction
a.
The Problem
Understanding and applying the major approaches to psychology are important skills to
develop in an introductory psychology course. Students need to be able to apply these
approaches throughout the semester as we compare how each psychologist would view the
topics of memory, personality, perception and abnormal disorders. Pilot assessment results
indicate that many students fail to encode some of basic information about the major
approaches and are unable to make inferences, compare beliefs and conceptualize concepts
about the applications of the approaches to various scenarios, suggesting incomplete schema
development. Because students may not choose or know how to engage higher order skills,
the process of encoding the information may not be as elaborative, and in depth, as it could
be.
b. The Purpose of this Study
The research study was designed to compare the effects of different instructional methods.
The variations in both methods were consistent with variations that the teacher or designer of
this class would normally engage in to improve learning for students in the class. The
purpose of the research was to formally evaluate whether the newer methods help to improve
student achievement compare to the previous practice. There will be no long-term follow-up
with these participants.
Students struggle with learning the six approaches to psychology more than any other topic
taught in the course. Each psychological approach finds different sources for behavioral
issues. Biological approach spends time associating chemical imbalances and neurological
issues with behavioral changes. Cognitive approach uses how people think to describe their
4
problems. Psychoanalytic searches the unconscious for behavioral answers. Humanistic
believes that to control one’s behavior they must have free will over their thoughts and
actions. Socio-cultural approach defines behavior of an individual by their upbringing. And
finally, behavioral approach studies how people can be rewarded and punished to change
their behaviors. They have few abilities to see that the same psychological problem can be
viewed, and treated, in more than one way. These same approaches come up again and again
throughout the year, thus students need a solid foundation with which to compare and
contrast these approaches. Prior to teaching the students the approaches to psychology in the
research study, students were taught how to use the program Inspiration and the concept
mapping process. A rubric, asking students to make connections using elaboration techniques
– such as cause and effect, examples and non-examples and contextualizing contributors, was
provided for the students in the previous chapter to teach the students how to generate a
concept map.
c. The Research Question
Many students never learn to compare, contrast and analyze the six approaches that
are taught to them in the first unit of the course. Students do well with recognition
and recall but don’t excel in the higher-order thinking skills. The following research
question was formulated to address the inability of students to use, compare, and
distinguish between the perspectives when asked to do so using a case study, and on a
multiple choice test, in the introductory unit of psychology:
i. Will the creation of concept maps improve students’ understanding,
demonstrated through proficiency in recall, analyzing the major themes in
each psychological approach, interpreting a case study based on one of the
5
psychological approaches and comparing the six major perspectives of
psychology, compared to using a standard worksheet to support learning?
II.
Review of Literature
a. Theoretical Justification
Organization and elaboration as learning strategies can be very helpful to improving
understanding, as both have been proven to improve recall and development of higher level
thinking skills (Gagne, 137). The six approaches to psychology are best studied in a case
study format, comparing how each of the major approaches would analyze and treat the same
psychological problem. For this task, students need recall knowledge of the basics of each
approach and must be able to analyze the major themes in each approach. The students with
deficiencies were not able to distinguish one approach from another causing them problems
with the multiple choice and essay portion of the test. Students could benefit by organizing
the approaches by each individual approach and describing the subsets that occur in each.
This organization, by approach, would allow the students to compare and contrast
approaches. Lack of organization can result in students having “trouble keeping track of all
the information” and making connections between the information (Gagne, 139).
Deshler & Lenz described a strategy as an individual’s approach to a task (cited in Ritchie
(1996, p.30). Students need these strategies to commit information to long-term memory, and
in turn, be able to use that information to develop higher-level concepts and support problem
solving. Information is often times presented to students without time for them to use selfgenerated or teacher generated strategies to improve retention (Ritchie, 1989, p. 30). To
improve learning, students can use elaboration as a strategy. Elaboration can occur through
paraphrasing, summarizing, drawing inferences or, even, generating a series of questions and
6
answers. E. Gagne (1985) discussed elaborations as logical inferences, images, continuations,
examples, details or anything else that helped the learner to connect the information, and thus,
be able to recall that information better at a later date (Derry, 21). Learners are also more
likely to form schema when they are analyzing examples and non-examples simultaneously
(Gagne, 165). These types of elaborations will help to organize the encoding and storage of
the approaches to psychology, and in turn, help with the application of the information when
students are asked to use the approaches to problem solve throughout the course.
b. Observational Justification
Students, throughout my tenure teaching the Introduction to Psychology class, have
been using a chart/worksheet as a study aid to learn about the six major perspectives
in psychology (See Appendix B). This chart, with pre-assigned row and column titles
from the teacher, is filled in by each student as the lecture on each approach is given
to them. Although this method works well with identification and recall of the six
approaches and their major themes, some students aren’t always able to compare and
contrast the approach to find the differences that make each unique. Some students
could benefit from an approach to learning this content that facilitates comparison,
contrast, and analysis of situations to a greater degree than the worksheets. Concept
mapping may help students to interact with the material to find examples and nonexamples among the perspectives and promote the deeper analytical processing
needed to complete the essay portion of their exam,
7
III.
Methods
a. Research Designs
The research study was designed to compare the effects two instructional methods. The
variations in both methods were consistent with variations that the teacher or designer of this
class would normally engage in to improve the class. The purpose of the research was to
formally evaluate whether a method that used concept mapping would help to improve
student achievement compare to the previous practice. There will be no long-term follow-up
with these participants.
Prior to teaching the students the six approaches to psychology in the research study, students
were taught how to use the program Inspiration and the concept mapping process. A rubric,
asking students to make connections using elaboration techniques – such as cause and effect,
examples and non-examples and contextualizing contributors, was provided for the students
while studying the previous chapter, to teach the students how to generate a concept map.
b. The Participants
Students were recruited to be subjects in this research study in the first week of the spring
semester during the Introduction to Psychology class. Students were given an assent/consent
form that was read aloud to them in class. The students who were 18 years of age were
permitted to sign their own assent/consent form. The students who were under 18 years of
age were asked to take the assent/consent form home with them to review again with their
parents.
All students received the same assent/consent form. The informed consent form was written
to describe that participation in the study was completely voluntary. There was no penalty of
8
any kind for a student who chose not to participate in the study or chose to remove
themselves from the study at any time.
Students did not sign the assent/consent form in front of the teacher. This was done at home
with the parent. Assent and consent were documented by the principal investigator.
Students over the age of 18 were able to sign the forms without parental permission. For
students under the age of 18. By not returning the form at all, a student’s data was not
included in the study results.
The principal investigator was assisted with the grouping and randomization process by the
school librarian. Students handed consent forms into the drop-box in the library. An Excel
Spreadsheet was used to form two groups of students. The librarian looked through forms
and created a list of consenting students. She was asked to make sure the groups were similar
in numbers, to which they were. The principal investigator got a copy of the students in their
assigned groups but did not get a copy of the students who had consented until the tests were
graded and data was ready to be reported.
c. The Materials
This study took place in a semester-long Introduction to Psychology course. Students
used a variety of materials throughout the unit to study the six major perspectives of
psychology. The students in the control group used the worksheets/maps provided,
textbook reading, laptop or desktop computers, and the online cloud based document
holder Google Drive. The students in the experimental group used the textbook
9
reading, laptop or desktop computers, and the program Inspiration. All students used
school issued computers, 1 per student.
Each day the investigator presented, a lecture to the entire class on one of the
perspectives. Students were then split into groups. The control group was given a
chart to fill in with information from the text. The information could be found in the
lecture notes or the test. Students were asked to use their notes from the class lecture
that day and to fill in the cells of the organized chart provided to them. All of the
information needed to complete the chart could be found in the lecture notes or
textbook. Even the information requested for the case study cells of the
worksheet/chart could be found by the students explicitly in their textbook. See
Appendix B for the chart.
The experimental group was given the starter concept map created with the program
Inspiration. With the starter map, students were asked to explain terms and concepts
and define principles taught to them about the psychological approach the class
focused on that day. Students were instructed to organize information into the map
from their daily notes and textbook reading. They were asked to expand on any type
of information that they found to be useful in distinguishing between the six
psychological approaches. Some of the information could be placed in predetermined categories already on the concept map created for them. Other
information needed to be added by the student, seeing connections between the other
approaches to psychology. Not all information on the concept map was created for
them to use. Students were asked to come up with their own blank areas to add to the
concept map. See Appendix C for the starter map.
10
Both groups were asked to add their work each day making comparisons to other
approaches and adding newly acquired information from the class lecture.
The test was based on the lecture notes, in-class discussion and textbook reading.
The test was a mixture of multiple choice and essay. The test was developed by the
researcher and had been used and validated using scores from a pilot assessment
administered to a similar Introduction to Psychology course in the fall of 2014. See
Appendix D.
d. Procedures
After obtaining consent from Penn State’s Internal Review Board, the research was
put in motion. It is important to note that all students taking the Introduction to
Psychology course were taught a lesson prior to the research lesson using the concept
mapping process. Students were taught how to use Inspiration and map using the
rubric provided, ensuring that all students knew how the program knowing that the
group assignment would be blind.
At the beginning of each class for the six days associated with the research
experiment, Students received direct instructions in the regular education classroom.
After approximately 15 minutes of direct instruction, the whole class of students was
taken to the library where they were split into two groups: experimental (A) and
control (B). It took approximately 3-5 minutes each day to transition the students
from lecture to individual work. There was a partition in the library that split the
computer lab from the laptop computers. The groups worked simultaneously. The
11
instructor was able to see and walk easily between the two groups, answering
questions. The following chart explains the daily procedures for both the control
group and the experimental group.
Experimental Group
Control Group
1. Introduce students to the idea of different theories of psychology
using the Elephant/Blind Men example.
a. Story presents itself that all of the blind
men are touching the same elephant (problem) but are all
coming up with different ideas of what it is (approaches to
psych)
b. Student/Teacher discussion on how multiple view points
of the origin of the problem can exist.
2. PowerPoint Presentation on Biological Approach to Psychology
for students with teacher led discussion and Q&A.
Day #1 –
Biological
Approach
12
3. Students will log-on to
the computers and open
the program Inspiration.
3. Students will log-on to
computers and open their
Google Accounts.
4. Students will open the
teacher’s pre-made
template on Inspiration
from the teacher’s
classroom website.
4. Students will access the
document on Biological
Approach that has been shared
with them on the teacher’s
classroom website.
5. Students will use the
template from the
teacher to map out the
basic terminology and
examples from the
Biological Approach to
Psychology.
5. Students will make their own
copy of the document into their
Google account and work on
completing the questions by
typing answers in the correct
spaces.
Students should include, but
not be limited to: visual
connections of beliefs,
examples and nonexamples, mapped cause
and effect, and
contextualizing
contributors to the
approach.
6. Students will save the
Inspiration document on
their H: drive to be used
in the next class period.
6. Students will share the
document with the teacher via
Google Drive.
1. PowerPoint Presentation on Psychoanalytic Approach to
Psychology for students with teacher led discussion and Q&A.
2. Students will log-on to
the computers and open
the program Inspiration.
Day #2 –
Psychoanalytic
Approach
3. Students will open the
3. Students will access the
teacher’s pre-made
document on Psychoanalytic Approach
template for the
that has been shared with them on the
Psychoanalytic
teacher’s classroom website.
Approach to
Psychology from the
teacher’s classroom
website.
Students should include:
examples, nonexamples, terminology,
proponents, and
treatment options.
4. Students will use the
template from the
teacher to map out the
basic terminology and
examples from the
13
2. Students will log-on to
computers and open their
Google Accounts.
4. Students will make their own
copy of the document into their
Google account and work on
completing the questions by typing
answers in the correct spaces.
Psychoanalytic
Approach to
Psychology.
Students should include, but
not be limited to: visual
connections of beliefs,
examples and nonexamples, mapped cause
and effect, and
contextualizing contributors
to the approach.
5. Students will save the
Inspiration document on
their H: drive to be used
in the next class period.
5. Students will share the
document with the teacher via
Google Drive.
1. PowerPoint Presentation on Behavioral Approach to Psychology
for students with teacher led discussion and Q&A.
Day #3 –
Behavioral
Approach
2. Students will log-on to
the computers and open
the program Inspiration.
2. Students will log-on to
computers and open their
Google Accounts.
3. Students will open the
teacher’s pre-made
template on Inspiration
from the teacher’s
classroom website.
3. Students will access the
document on Biological
Approach that has been shared
with them on the teacher’s
classroom website.
4. Students will use the
template from the
teacher to map out the
basic terminology and
examples from the
Biological Approach to
Psychology.
4. Students will make their own
copy of the document into their
Google account and work on
completing the questions by
typing answers in the correct
spaces.
Students should include, but
not be limited to: visual
14
connections of beliefs,
examples and nonexamples, mapped cause
and effect, and
contextualizing
contributors to the
approach.
5. Students will save the
Inspiration document on
their H: drive to be used
in the next class period.
5. Students will share the
document with the teacher via
Google Drive.
PowerPoint Presentation on Humanistic Approach to Psychology
for students with teacher led discussion and Q&A.
Day #4 –
Humanistic
Approach
2. Students will log-on to
the computers and open
the program Inspiration.
2. Students will log-on to
computers and open their
Google Accounts.
3. Students will open the
teacher’s pre-made
template on Inspiration
from the teacher’s
classroom website.
3. Students will access the
document on Humanistic
Approach that has been shared
with them on the teacher’s
classroom website.
4. Students will use the
template from the
teacher to map out the
basic terminology and
examples from the
Humanistic Approach to
Psychology.
4. Students will make their own
copy of the document into their
Google account and work on
completing the questions by
typing answers in the correct
spaces.
Students should include, but
not be limited to: visual
connections of beliefs,
15
examples and nonexamples, mapped cause
and effect, and
contextualizing
contributors to the
approach.
5. Students will save the
Inspiration document on
their H: drive to be used
in the next class period.
5. Students will share the
document with the teacher via
Google Drive.
1. PowerPoint Presentation on Cognitive Approach to Psychology
for students with teacher led discussion and Q&A.
Day #5 –
Cognitive
Approach
2. Students will log-on to
the computers and open
the program Inspiration.
2. Students will log-on to
computers and open their
Google Accounts.
3. Students will open the
teacher’s pre-made
template on Inspiration
from the teacher’s
classroom website.
3. Students will access the
document on Cognitive
Approach that has been shared
with them on the teacher’s
classroom website.
4. Students will use the
template from the
teacher to map out the
basic terminology and
examples from the
Cognitive Approach to
Psychology.
4. Students will make their own
copy of the document into their
Google account and work on
completing the questions by
typing answers in the correct
spaces.
Students should include, but
not be limited to: visual
connections of beliefs,
examples and non-
16
examples, mapped cause
and effect, and
contextualizing
contributors to the
approach.
5. Students will save the
Inspiration document on
their H: drive to be used
in the next class period.
5. Students will share the
document with the teacher via
Google Drive.
1. PowerPoint Presentation on Socio-Cultural Approach to
Psychology for students with teacher led discussion and Q&A.
Day #6 –
Socio-Cultural
Approach
2. Students will log-on to
the computers and open
the program Inspiration.
2. Students will log-on to
computers and open their
Google Accounts.
3. Students will open the
teacher’s pre-made
template on Inspiration
from the teacher’s
classroom website.
3. Students will access the
document on Socio-Cultural
Approach that has been shared
with them on the teacher’s
classroom website.
4. Students will use the
template from the
teacher to map out the
basic terminology and
examples from the
Socio-Cultural
Approach to
Psychology.
4. Students will make their own
copy of the document into their
Google account and work on
completing the questions by
typing answers in the correct
spaces.
Students should include, but
not be limited to: visual
connections of beliefs,
examples and non-
17
examples, mapped cause
and effect, and
contextualizing
contributors to the
approach.
Day #7 Assessment
5. Students will save the
Inspiration document on
their H: drive to be used
in the next class period.
5. Students will share the
document with the teacher via
Google Drive.
1. Students will take the
criterion assessment, in
a 41 minute class
period, using a paper
and pencil test. This test
will be used for study
instrument only.
1. Students will take the criterion
assessment, in a 41 minute
class period, using a paper and
pencil test. This test will be
used for a study instrument
only.
When students finished, they were asked to print their concept maps and charts to
study with later. Many students in the control group chose to use the online version
of their chart to study. The experimental group did not have digital access. Their
document was static.
IV.
Analysis and Results
a. Research Question
Will the creation of concept maps improve students’ understanding, demonstrated
through proficiency in recall, analyzing the major themes in each psychological
approach, interpreting a case study based on one of the psychological approaches
and comparing the six major perspectives of psychology, compared to using a
18
standard worksheet to support learning?
b. Descriptive Statistics
The descriptive statistics, are presented in the Tables below. For a complete list of
experimental statistics, see Appendix D.
Table 1: Essay Test Results
Group
Control Group
Experimental Group
N
25
25
Mean
23.96
21.64
Standard Deviation
5.42
7.96
Mean
17.8
16.88
Standard Deviation
3.13
3.07
Total points for the essay = 30
Table 2: Multiple Choice Test Results
Group
Control Group
Experimental Group
N
25
25
Total points for the multiple choice = 24
Table 3: Combined Multiple Choice and Essay Results
Group
Control Group
Experimental Group
N
25
25
Mean
20.88
19.26
Standard Deviation
5.37
6.43
c. Analysis of Results
Microsoft Excel was used to perform independent samples t-tests to compare the means of
the experimental and control groups. The result indicated no significant effects between the
treatments on performance in all of the tests. The p-value for the essay test was .23. The pvalue for the multiple choice test was .30. And the p-value for the combined exams was .19.
19
Thus, the hypothesis that states that there would be a significant difference between the
experimental group and control group can be rejected. The students using the teacherorganized worksheet did not score lower than the students given the ability to create a
concept map using Inspiration. Student performances on the tasks of recall, analysis of
themes in theories, and comparisons of approaches were no better for students in the
experimental group than in the control group. From this, we can conclude that the concept
map activities used to learn the six approaches to psychology did not improve one’s ability to
compare and contrast, analyze and interpret when compared to the worksheet chart activity
that was used.
An additional analysis was done using the scores from the most difficult test questions.
Those questions were singled out and focused on to see if there was any significant difference
between the control group and experimental group. No significant difference was found
between the two groups for the questions with high difficulty. See Table 4.
Table 4: Questions Marked Difficult
Group
Control Group
Experimental Group
p-value
3.42
3.03
Standard Deviation
1.39
1.23
Finally, each test question was analyzed for skills needed to answer the questions
successfully. These required skills were then compared with cognitive activities facilitated by
the experimental and control treatments. A prediction was then made whether performance
on each test question should be enhanced by the treatments. There was no single advantage
to one group for 18 of the 25 test questions. There was a predicted advantage to the
experimental treatment for questions 7, 8, 10, 11, 16 and 21, and a predicted advantage to the
control treatment for question 15. See the breakdown of the questions in Appendix Table F.
20
A t-test was performed to compare performance of experimental and control groups on
questions 7, 8, 10 11, 16, and 21, and indicted there was no significant difference in
performance between the groups on this set of questions.
Table 5: Questions Labeled With a Predicted Advantage to Treatment Questions
Group
Control Group
Experimental Group
V.
p-value
3.95
3.33
Standard Deviation
1.84
1.37
Discussion
When comparing the results from students who learned using the traditional worksheet to the
results of students who used the concept mapping mindtool, the data supports conclusion that
neither the worksheet activity nor the concept mapping activity was more effective in
improving students’ recall, analyzing the major themes in each psychological approach,
interpreting a case study based on one of the psychological approaches and comparing the six
major perspectives of psychology. Students in the experimental group using the concept map
were able to create a visual representation of the lecture data by adding information about
important individuals, ideas and examples to their maps each day. Students in the control
group were asked to recall data and place it into a teacher created table, adding to the chart
each day.
Unexpectedly, the concept maps submitted by the students in the experimental group did not
demonstrate the cognitive activities associated with the example, non-example process. It is
possible that the rubric created for the experimental group did not promote the cognitive
activities required to succeed on the test. The rubric needed to be directly tied with the test,
21
creating the rubric based on the test. Examples of these concept maps can be found in
Appendix G.
Many students create the same non-example for all approaches. Students in the experimental
group were asked to expand on their teacher-created map as they learned new, connecting
information. It is necessary to point out that many of the students chose to do the bare
minimum to fulfill the requirement, not expanding past the point of information given to them
on the teacher-created concept map, lecture and reading.
When assessing skills practiced on the chart versus the concept map, there were no different
requirements for either of the groups. The potential for difference lies in the inferences
needed by the experimental group to create examples and non-examples on the concept map.
Students working on the concept map made very few connections besides the ones that were
originally mapped out for them. There was little comparison, after review of the student work
submitted for the experimental group, between the approaches to psychology. Students
simply picked another category, or approach, and used it as the non-example required. It was
expected that students would create non-examples using all other approaches for each of the
non-examples.
In terms of an advantage, both groups were asked to complete similar tasks but the
experimental group should have made more compare and contrast connections. Viewing the
data collected, this was not the case. Therefore, no significant difference was found.
The only potential advantage could have been to the control group with the teacher created
worksheet/chart. Each group was asked to recall information and connect it with the six other
22
approaches. The disadvantage to the experimental group was the lack of structure to the
concept map. Students had to recall the information and make their own structure and
connections. Had the students created a full concept map with examples and non-examples
completely filled out, this would have been an advantage to the experimental group. After
finding that the students were missing information in their concept maps, it was no surprise
that there was no advantage to this group. The control group had the chart columns filled in,
directing them to what was important and, therefore, what to focus on writing.
VI.
Limitations of the Study
There was a limitation that occurred during the experiment. Although the students’ consent to
have their data used in the experiment was blind to the principal investigator, scores were not
given to the students blindly. The teacher was aware of the students name on the work of the
test when it was being scored. This could be rectified by having students use numbers instead
of names on the tests.
VII.
Implications and Conclusions
This research study shows that worksheets can be just as beneficial as the type of mapping
students did in the study.
Suggestions for future studies:
1. Students must be very familiar with the technology being used before they are asked to
work with it to facilitate analysis and comparisons.
2. Students must be shown how to make connections and must be given examples of
effective maps.
3. Teacher should score the concept maps using the rubric presented to the students.
4. Teacher should read and score the test blindly – unaware of the students’ identities.
23
5. Principal investigator should do a close analysis of the requirements of the test and the
cognitive activities promoted by the requirements of the rubric activities in order to be
predictive with the rubric, not reflective.
24
VIII.
Appendices
Appendix A: Consent for Research Form
25
26
Appendix B: Chart Worksheet – Control Group
Appendix C: Example Concept Map Starter – Experimental Group
27
Appendix D: Lesson Exam
28
29
Appendix E:
Appendix F:
Question #
1
2
3
4
5
6
7
8
9
10
30
Group Benefitting
Both
Both
Both
Both
Both
Both
Experimental
Experimental
Both
Experimental
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
Appendix G:
31
Experimental
Both
Both
Both
Control
Experimental
Both
Both
Both
Both
Experimental
Both
Both
Both
Both
32
Appendix H: Concept Map Rubric
33
34