1
MEASURING THE IMPACT OF SERVICE-LEARNING IN HIGHER EDUCATION: HOW
PERSONAL SET CHARACTERISTICS, ACADEMIC PERFORMANCE, AND CIVIC
ENGAGEMENT INTERACT IN THE PRESENCE OF SERVICE-LEARNING
A Research Article Funded By Kansas Campus Compact
2
3
ABSTRACT
Research dating back to Astin & Sax (1998) and Eyler, Giles, and Braxton (1997) has
shown a clear connection between service-learning pedagogy and development of students’’
personal set characteristics, increased civic engagement, and improved academic performance.
This paper explores how those three factors interact in the presence of service-learning.
Specifically, it considers whether a relationship between service-learning and increased personal
set characteristics leads in turn to increased civic engagement and academic performance, or
whether the three factors act independently in the presence of service learning. This study, which
focused on higher education in Kansas, found that the three factors act independently in the
presence of service learning. It found service learning had no impact on personal skill set,
marginal negative impact on civic engagement, and statistically significant negative impact on
academic performance. This paper considers possible explanations for these unexpected findings
and recommends direction for further research in this area.
4
Literature Review
From the long heritage of research on how service-learning impacts personal skill set,
academic performance, and civic engagement we know that service-learning can have a positive
effect on students’’ personal set characteristics development. (Astin & Sax, 1998; Eyler, Giles, &
Braxton, 1997; Eyler & Giles, 1999). We also know that service-learning can have a positive
impact on students’’ academic performance (Astin & Sax, 1998; Eyler & Gile, 1999; Eyler, Root,
& Giles, 1998). We furthermore know that service-learning increases civic engagement in terms
of social responsibility and citizenship skills (Astin & Sax, 1998; Eyler & Giles, 1999; Eyler,
Giles & Braxton, 1997).
What is not known is how personal development, academic learning, and civic
engagement interact in the presence of service-learning. Research outside the realm of servicelearning has found that specific personal set characteristics are valid predictors of strong
academic performance (Bandura, 1986; Shell, Colvin, & Bruning, 1995). Based on that premise,
this study considers whether service-learning changes the personal set characteristics linked to
strong academic performance, and this in turn results in increased civic engagement and
improved academic performance. It also explores whether improved academic performance or
increased civic engagement can be seen even in the absence of change in the personal set
characteristics.
This review looks at the existing research for each of the three areas: personal set
characteristics, academic performance, and civic engagement. This provides a research
foundation for each of the three areas independently. The literature review includes discussion of
how research in each of these three areas relates to each other.
5
Personal Set Characteristics
The personal set characteristics associated with learning and strong academic
performance have been documented in a long history of studies. Personality researchers have
argued that personality traits account for a significant portion of variance in academic
performance (Chamorro-Premuzic & Furnham, 2003; Duff, Boyle, Dunleavy, & Ferguson, 2004;
Furnham, Chamorro-Premuzic, & McDougall, 2003; Komarraju & Karau, 2005; Marsh et al.,
2006; Martin, Montgomery, & Saphian, 2006). From this research have emerged five basic
personality factors, known as the big-five personality traits factors (McCrae & John, 1992): (1)
Extraversion, (2) Agreeableness, (3) Conscientiousness, (4) Emotional Stability, and (5)
Openness to Experience. Further studies have found a positive correlation between two of the
big-five personality measures and academic achievement: openness and conscientiousness
(Costa & McCrae, 1992; Diseth, 2003; Piedmont, 1998). The other major traits have shown less
consistent associations with academic achievement (Martin et al., 2006).
Individuals high in conscientiousness are typically efficient, organized, reliable,
responsible, persistent, thorough, and goal directed. Digman (1989) indicated that
conscientiousness is the one of the big-five model that is most closely linked to a will to achieve.
It was also the trait that served as the strongest predictor of academic performance, even after
controlling for intelligence (Porpat, 2009).
McCrae and John (1992) described people high in openness as being artistic, curious,
imaginative, insightful, original, and as having wide interests. Numerous studies have found
openness to be a major correlate of academic achievement and success (Asendorph & Van Aken,
2003; Blickle, 1996; De Raad & Schouwenburg, 1996; Paunonen & Ashton, 2001), and learning
motivation (Tempelaar, Gijselaers, Schim Van Der Loeff, & Nijhuis, 2007).
6
Beyond the big-five personality traits, research has also linked self-efficacy and empathy
to improved academic performance. Studies that looked at self-efficacy can be traced back to the
foundational work of Bandura (1986) that defined self-efficacy as a person’’s ““judgment of their
capabilities to organize and execute courses of action required to attain designated types of
performances”” Bandura’’s theory holds that the beliefs people have about themselves influences
the way they behave (Bandura, 1986). This extends to students in that those who believe they can
do well experience higher motivation in terms of effort, persistence, and behavior (Pintrich &
Schunk, 2002). Additional studies have found a relationship between high self-efficacy and
strong performance on standardized tests (Shell, Colvin, and Bruning, 1995), writing
performance (Bissell and Collins, 2001), and task persistence (Bouffard-Bouchard, 1990) .Self-efficacy
has also been found to be positively related to deep processing of subject material and negatively
related to surface processing (Seifert & O’’Keefe, 2001), which indicates that higher self-efficacy
might increase critical thinking.
The fourth trait that research has consistently linked to improved academic performance
is empathy, one’’s vicarious experience of another’’s emotional experience——or feeling what the
other person feels (Mehrabian & Epstein, 1972). Bonner and Aspy (1984) discovered that
students with strong GPAs (a measure of knowledge) were best able to construct empathic
messages and responses. Stroessner, Beckerman, and Whittaker (2009) found students in a
history class who participated in a role playing game exhibited greater empathy for the historical
figures and also improved academic skills in the course. Pavey, Greitemeyer, and Sparks (2012)
found that empathetic arousal is a strong predictor of helping behavior. This might be especially
important in the current study in that empathy might interact with academic achievement more in
a service-learning course than in more traditional courses. If students understand the receiver’’s
7
need for the service (empathy), they might put more effort into learning the concepts needed to
complete the service.
In view of these findings from previous research, this study measured changes in four
personal skills characteristics: Openness, conscientiousness, self-efficacy, and empathy. Research
has linked all four to improved academic performance. All four are also traits commonly seen in
students who demonstrate high civic responsibility.
Civic Engagement
Specific definitions of civic engagement vary from study to study, but all those
definitions share certain common characteristics. The two most common characteristics
described in most civic engagement studies are good citizenship and civic responsibility. Just like
civic engagement, good citizenship and civic responsibility are defined in different ways by
different authors and even sometimes interchangeably (Williams, 2000), hence making it hard to
review without inclusion of the authors’’ biases. In terms of good citizenship, Walzer (1970) held
that citizens come in kinds and degrees and that all men and women legally bound to the state are
not in fact morally bound in the same way. Barber (1998) coined the terms thin democracy and
strong democracy to describe this phenomenon of citizens being morally bound in different
ways. Perry and Katula (2001) developed these ideas further with a definition of good
citizenship along three dimensions: an individual’’s motivations and skill, philanthropic and civic
behaviors, and political behaviors. Battistonia (2002) developed a more detailed conceptual
framework of good citizenship based on seven criteria: civic professionalism, social
responsibility, social justice, connected knowing, public leadership, public intellectual and
engaged/public.
8
In describing civic responsibility, Degelman (2000) holds that civic responsibility is not
an intuitive process and therefore young people must learn to participate in democracy.
Degelman’’s framework for measuring civic responsibility is based on three criteria: Citizenship
for democracy, participatory democracy and social responsibility. Gottleib and Robinson (2004),
similarly describe civic responsibility in the context of social responsibility, civic engagement
and community involvement.
Given these intertwined definitions of good citizenship and civic responsibility, this paper
treats the two variables as the same thing and as two of the most common described attributes of
civic engagement. . This research particularly focuses on the impact of service-learning in
changing self-efficacy and emotional empathy as it relates to increased civic engagement.
Eyler, Giles, and Braxton (1997) foundational study of 1,500 students in 20 colleges and
universities showed that students who chose service-learning demonstrated more developed civic
engagement than students who did not choose service-learning. Additionally, they found that
participation in service-learning a student’’s self- efficacy. They showed service-learning to be a
predictor of student’’s political participation skills, tolerance for others, empathy, and ability to
remain open to new ideas. The work of Eyler, Giles, and Braxton (1997) shows the
interconnectedness of increased personal set characteristics and increased civic engagement and
that both are enhanced by service-learning.
Perry and Katula’’s (2001) review of 37 empirical studies found a consistent and
significant increase in individual self-efficacy and motivation following a service-learning
project, directly related to the individual’’s community connectedness and understanding of local
social issues. Their review found that giving and volunteerism tended to increase in most of the
service-learning studies analyzed, but it was not clear whether volunteerism would be sustained
9
over a period of time. However, the authors ound no significant relationship between servicelearning and voting behavior, and made no mention of a relationship service and increased civic
engagement or improved academic performance.
Hamilton and Zeldin (1987) found students engaged in service-learning posted greater
increases in political efficacy than student not involved in service learning. The work of
Youniss, McLellan, and Yates (1997) followed students for 15 years or more into adulthood and
found that students who participated in high school government or community service projects
were more likely to vote or join community organizations as adults. This supports the view that
exposure to service-learning at a young age can help build civic identity and a sense of social
responsibility for the community’’s well-being (Youniss, McLellan, & Yates, 1997). Melchior
(1999) and Berkas (1997) learned that students who engaged in high quality service-learning
programs showed an increase in the degree to which they felt aware of the community needs and
their belief in making a difference for that community. Marby (1998) found that service-learning
was most effective as a civic and academic pedagogy when students reported at least 15 to 20
hours of service and had sufficient interaction and reflection with supervisors. The work of
Hunter and Brisbin (2000) showed that students who participated in service-learning tended to be
more civically engaged.
The foundational work of Astin and Sax (1998) found an increase in civic engagement as
measured changed in civic responsibility indicators as a result of service-learning. They
measured 12 civic responsibility outcomes related to students’’ commitment to action with regard
to social problems and social justice. The results indicated that all the 12 civic responsibility
outcomes were positively influenced by service participation with the majority of the outcomes
having significantly different coefficients (Astin & Sax, 1998). Aslam, Jaffery, and Zaidi (2011)
10
used the civic responsibility survey developed by Andrew Furco (1998) in a study of 13 college
girls who volunteered for community health projects for 26 hours during the spring semester and
showed statistically significant improvements on measures of connection to the community, civic
awareness and altitude, and civic action and efficacy (Aslam, 2011). Here again, the positive
correlation between civic engagement and efficacy as a result of service-learning can be seen.
These studies show a clear relationship between service-learning experiences and
increased civic engagement. In some studies, an association is also seen between civic
engagement attributes and the personal skill-set characteristic of efficacy. This study will extend
that research by looking at the interaction among improved personal set characteristics, improved
academic performance, and increased civic engagement. Specifically this study will look at
whether increased civic engagement occurs, as seen in previous studies, even in the absence of
improved personal characteristics.
Academic Performance
Research findings on the relationship between academic performance and service
learning are mixed, with studies supporting each side of the question. Findings showing a
relationship between service-learning and academic performance stem back to the work of Astin
and Sax (1998) that found the effect of service-learning on grade point average was small ––
about 0 .1 grade points –– but statistically significant. Among students in the study, 69% of those
who had at least a B+ grade average in high school, were able to maintain that average in
college. That compares to just 56% of similar students who had not participated in service.
Gray’’s (1998) study of Learn and Service America Higher Education institutions found slightly
higher grade point averages and more course satisfaction among students in service-learning
11
classes, with the greatest improvement among students who volunteered more than 20 hours per
semester.
Vogelgesang and Astin’’s (2000) quantitative longitudinal study of 22,000 students found
that both course-based service-learning and generic community service were associated with
higher college GPA, but the effect on GPA and writing was even stronger among service-learning
students.
Research finding no relationship between academic performance and service-learning
stems back to the foundational work of Boss (1994) that looked at the effect of community
service on academic performance. The study showed students in the community service section
did a better job of defining issues and were using more principled moral reasoning, but found no
difference in course grades(Boss, 1994).
The work of Miller (1994) looked at the effects of service learning when service was not
a class requirement, but optional. In this study service-learning students reported higher
expectations for the class but their final grades did not differ from students who chose the nonservice option. Likewise Hudson (1996) and Kendrick (1996) found no statistically significant
difference in academic performance, although service-learning students in the Kendrick (1996)
study demonstrated improved ability to apply course concepts to new situations.
Parker-Gwin and Mabry (1998) studied 260 students participating in three models of
service learning found that when service is required rather than optional and especially when
reflection is limited, service-learning can cause a decline in students’’ view of the importance of
community service. They found improved academic performance only when students
participated in service learning voluntarily and who had the opportunity to reflect on their service
and its connections to the course content(Parker-Gwin & Mabry, 1998).
12
The wide range of findings in the literature may have to do with sample size. In the larger
studies academic benefits were more pronounced. In smaller studies it was difficult or
impossible to see any academic benefit from student learning. Additionally, studies that found
no academic benefit tended to rely on students’’ self-reported perception of their academic
growth. By contrast, studies that found an academic benefit tended to rely on objective measures
such as course grade or semester grade point average.
This study relied heavily on objective measures of academic performance such as course
grade and grade point average. It used a model to predict expected course grade and compared
expected grades to actual grades. The prediction of student outcomes based on current and other
previous academic performance has been studied extensively over the past 40 years. Eskew and
Faley (1988) introduced the first full mathematical model for predicting performance in their
study that looked at the first college-level financial accounting course. They found 54% of the
variance in course examination performance could be explained by ability as measured by SAT
scores, course effort as measured by the number of quizzes taken, previous and current academic
performance as measured by college and high school grade point average and previous
accounting or related experience. Of these factors, they found aptitude and effort to be dominant
(Eskew & Faley, 1988).
Since then researchers have tried variations of the Eskew & Faley (1998) model and
applied the model to a variety of classes, all with similar results. Borde (1998) applied the model
to undergraduate marketing courses and found grade point average to be the strongest predictor
of performance. Borde’’s model did not include an independent variable for effort(Borde, 1998).
Kalbers and Wienstein (1999) applied a more detailed model to introductory accounting classes
but found once again that college academic performance (GPA) and aptitude (ACT/SAT score)
13
were the most prominent characteristics for explaining student performance. Kruck and Lending
(2003) applied the Eskew and Faley (1988) model to an introductory information systems class
and found motivation and GPA predict performance.
D’’Souza and Maheshwari (2010) applied the Eskew and Faley (1988) model to an
introductory management science course. They found grade point average, average homework
score, attendance, and completion of a pre-calculus prerequisite to be the most important
predictors.
Therefore, it is consistent from literature review that t the key independent variables in
predicting student course academic outcomes are aptitude as measured by SAT scores,
preparation as measured by current grade point average and high school grade point average, and
effort as measured in any of a variety of ways. This study relies on a model for expected
academic performance based on current grade point average, high school grade point average,
and student motivation as measured by percentage grade on homework assignments. By
controlling for these factors this study will measure the effect of service learning on academic
performance.
Research Questions
This quantitative study seeks to extend existing research by examining the relationship
between service-learning and personal skill set, academic performance, and civic engagement.
In particular, this exploratory study looks at whether there is a relationship between servicelearning and changes in the personal skill set, and that those changes in turn relate to changes in
academic performance or civic engagement, or whether the three areas react independently to the
presence of service learning.
14
RQ1: Is there a relationship between service-learning and improved levels of personal set
characteristics (or skills)?
a. Hypothesis R1: There is a relationship between service-learning and improved
levels of personal set characteristics (or skills).
b. Hypothesis R0: There is no relationship between service-learning and improved
levels of personal set characteristics (or skills).
RQ2: Is there a relationship between service-learning and improved civic engagement even in the
absence of improved personal set characteristics?
a.
Hypothesis R1: There is a relationship between service-learning improved civic
engagement even in the absence of improved personal set characteristics.
b.
Hypothesis R0: There is no relationship between service-learning improved civic
engagement even in the absence of improved personal set characteristics.
RQ3: Is there a relationship between service-learning and improved academic outcomes even in
the absence of improved personal skill characteristics?
a. Hypothesis R1: There is a relationship between service-learning and improved
academic outcomes even in the absence of improved personal skill characteristics.
b. Hypothesis R0: There is no relationship between service-learning and improved
academic outcomes even in the absence of improved personal skill characteristics.
Methodology
This quantitative study looked at the relationship among changes in personal skill set,
changes in civic engagement, and improved academic performance in the presence of servicelearning. The study used a pre-test and a post-test to measure changes in personal set
characteristics and civic engagement. It used data collected from students, teachers, and
15
institutions to measure academic outcome for the target course against expected outcomes based
on aptitude, ability, and effort.
Participants
Participants for this study were recruited from Midwestern institutions of higher learning
with the help of Kansas Campus Compact personnel. It initially included 364 students from eight
institutions of higher education in Kansas. Ultimately 139 students completed both a valid pretest and a valid post-test. Of those 44 students completed full grade and academic background
information to be included in the study of changes in academic performance. Institutions
participating in the study included state universities, private universities, and community
colleges.
Study participants ranged in age from 18 to 57, with the average age 22 and the median
age 20. Of those, 77 (55%) were female and 60 (43%) were male. Thirty-seven percent were
first-year students, 32% were sophomores, 16% were juniors and 15% were seniors.
The study included 80 (58%) students who were currently participating in servicelearning classes and 59 (42%) who were currently in traditional classes. Of all students in the
study, 62 (45%) reported having previously participated in at least one class where service was a
part of the course requirements. Additionally, 89 (64%) of the students reported having
participated in volunteer work in the past. There was wide variation in the kind of volunteer
work reported.
Procedures
Data for this study was gathered from February 2012 through June 2012. Students in the
study took a pre-service survey during the first three weeks of the spring semester and took the
post-service survey during the last three weeks of the spring semester. Both the pre- and post-
16
service surveys were administered in the classrooms using an online survey instrument.
Individual students were identified by student ID number so that pre- and post-test results could
be compared without compromising the students’’ privacy. Field testing determined that the
survey took approximately eight minutes to complete electronically. In instances where
computer access was not available in the classroom, students completed a paper copy of the preand post-service tests. SPSS was used to analyze the final data.
In addition to the pre- and post-service surveys, student academic data was collected from
the students, from their instructors, and from their institutions. Specifically this study collected
ACT scores, college grade point averages, and high school grade point averages, from the
students and from the institutions. It also used course grades and homework grades for the
course, both of which were collected from participating instructors.
The pre- and post-service survey included 10 questions from The Big Five Inventory
(Rammstedt & John, 2007) to measure changes in personal set. Students rated themselves using a
5-point Likert scale. Rammstedt and John (2007) tested this abbreviated version of the full 44question survey and found that although the effect sizes were lower for the 10-item, the results
were still accurate and sufficient if time was limited.
The pre- and post-service survey included 16 questions from The Toronto Empathy
Questionnaire (Spreng, McKinnon, Mar & Levine, 2009) to measure changes in empathy.
Students rated themselves using a 5-point Likert scale. This instrument was found by Spreng, et
al. (2009) to have strong validity and high test-retest reliability. It’’s brevity and high reliability
and validity made it ideal for this study.
The pre- and post-service survey included 20 questions from The College Self-Efficacy
Inventory (Solberg, O’’Brien, Villareal, Kennel & Davis, 1993) to measure changes in self-
17
efficacy. Students rated themselves using a 10-point Likert scale. Self-efficacy is domain
specific, related to specific tasks (Barry & Finney, 2009). The instrument selected for this study
specifically measures a person’’s self-efficacy in the realm of college life and academics. Barry
and Finney (2009) found the measure to be both valid and reliable in measuring a student’’s
confidence in various aspects of college.
The pre- and post-service survey included 24 questions from The Civic Responsibility
Survey (Furco, Muller, & Ammon, 1998) to measure changes in civic responsibility. This study
used the Level 3 version of the survey that shows the highest overall reliability for high school
students. Although not originally piloted for college students, the civic responsibility survey has
recently been used for this level of students as well (Aslam et al, 2011).
The pre- and post-service surveys collected information from students regarding college
grade point average, high school grade point average, and ACT score. This information was
cross-referenced against college grade point average, high school grade point average and ACT
score provided by the institution. College and high school grade point average were used to
measure student ability. Standardized test data was not used in the data analysis for this study
because only a small percentage of survey participants had taken either the SAT or ACT test .
The course grade for homework for each student was collected from the participating instructors
as a measure of effort. This follows the general model first established by Eskew and Faley
(1988) and validated by several studies that replicated their results. (Borde, 1998; Kalbers &
Wienstein, 1999; Kruck & Lending, 2003; D’’Souza & Maheshwari, 2010). The resulting
formula for expected course grades was:
SUM = (HSGPA + CGPA + HW) / 3
SUM = Expected course grade
HSGPA = High school GPA
18
CGPA = College GPA
HW = Homework score
Participating instructors provided end of course grades for each student. These were
compared to the grades derived from the formula for expected course grades. Analysis showed a
statistically significant correlation between expected course grade derived from the model and
actual grade, validating the model for expected course. Course grades were slightly higher than
expected with the mean for course grade at 3.118 compared to the mean for expected course
grade at 2.67.
One-Sample Statistics
N
Mean
CG
ECG
44
44
3.1182
2.6741
Std.
Deviation
1.62988
1.11179
Std. Error
Mean
.24571
.16761
Results
This study found no relationship between service-learning and personal set characteristics
or service-learning and civic engagement. The study found a statistically significant negative
relationship between service learning and academic performance. Following are details of these
unexpected results in each of the three subject areas.
Personal Set Characteristics
For the first research question, is there a relationship between service-learning and
improved levels of personal set characteristics, the null hypothesis was supported. An ANOVA
analysis found no statistically significant relationship between service learning and personal set
characteristics. (Table 1) Students in service-learning courses did not see greater increases in
personal set characteristics than did students in non-service learning courses.
19
Table 1
Mixed ANOVA Results for Students’ Personal Characteristic Scores and Service-learning
Variable
df
F
Ș2
p
Self Efficacy
Empathy
Openness
Conscientiousness
1
1
1
1
.151
.800
.286
1.218
.001
.006
.009
.002
.699
.373
.272
.272
For two of the personal set characteristics——openness and conscientiousness——mean
scores for students enrolled in the non-service-learning courses increased more than mean scores
for students in the service-learning courses. (Table 2)
Table 2
Mean Scores for Students’ Personal Set Characteristics
Trait
Class Type
Pre-Test Mean
Post-Test Mean
Self Efficacy
Service-learning
Non-Service-learning
152.959
150.000
157.257
152.759
Empathy
Service-learning
Non-Service-learning
62.205
65.526
62.244
64.596
Openness
Service-learning
Non-Service-learning
3.513
3.364
3.450
3.483
Conscientiousness
Service-learning
Non-Service-learning
3.906
3.983
3.931
4.068
20
Analysis of specific statements asked in survey questionnaires found significant
differences between pre and post-test means for three questions. These differences were
significant at more than a 95% confidence interval as shown in Table 3.
Table 3
Mean Scores for Significantly Different Statements Related To Personal Set Characteristics
Statement
Trait
Pre-Test Mean Post-Test Mean
1.When someone is
excited, I tend to get
excited too.
Empathy
3.582
2.When I see someone
being mistreated unfairly,
I do not feel much pity
for them.
Empathy
3.Make new friends
at college.
Civic Engagement
Self-Efficacy
3.824
Sig (T-test)
0.030
1.774
1.938
0.037
7.735
8.113
0.040
For the second research question, is there a relationship between service-learning and
improved civic engagement even in the absence of improved personal set characteristics, the null
hypothesis was supported. An ANOVA analysis found no statistically significant relationship
between service learning and improved civic engagement. (Table 3) Students in service-learning
courses did not see greater increases in civic engagement than did students in non-service
learning courses.
Table 4
Mixed ANOVA Results for Students’ Personal Set Characteristic Scores
Variable
df
F
Ș2
p
Connection to Community
Civic Awareness
Civic Efficacy
1
1
1
.641
1.910
1.187
.005
.014
.009
.425
.169
.278
21
In every factor of civic engagement, mean scores for students in non-service-learning
courses increase more than mean scores for students in service-learning courses. (Table 5)
Table 5
Mean Scores for Civic Engagement
Trait
Class Type
Pre-Test Mean Post-Test Mean
Connection to
Community
Service-learning
17.51
Non-Service-learning 17.80
17.28
18.05
Civic Awareness Service-learning
42.75
Non-Service-learning 41.63
42.81
43.44
Civic Efficacy
39.88
39.47
Service-learning
39.59
Non-Service-learning 37.67
Analysis of specific statements asked in survey questionnaires found significant
differences between pre and post-test means for three questions. These differences were
significant at more than a 95% confidence interval as shown in Table 6.
Table 6
Mean Scores for Significantly Different Statements Related To Civic Engagement
Statement
Trait
Pre-Test Mean Post-Test Mean
Sig (T-test)
I often try and act
on political, social or
national problems in
the community.
Civic awareness
3.399
3.651
0.019
I try to find time or a
way to make a positive
difference in the
community.
Civic Efficacy
4.049
4.232
0.038
I understand how
political, and social
issues or policies
affect members in a
community.
Civic Awareness
4.055
4.195
0.057
22
Academic Outcomes
For the third research question, is there a relationship between service-learning and
improved academic performance even in the absence of improved of personal set characteristics,
the null hypothesis was supported. Although the study showed that a relationship exists it was
found to be a negative correlation between service learning and course grade, significant at the
.05 level. (Table 7) The mean course grade for service learning students was 2.64 compared to a
mean course grade of 3.48 for non-service learning students. (Table 10 –– Table 11) A step
regression found that service learning accounted for 21.6% of course grade variance. (Table 8)
Table 7
Pearson Correlations Related to Academic Performance Model
SL
CG
HCG
GPA
HSGPA
ECG
Pearson Correlation
SL
1
N
44
Pearson Correlation
-.257
N
44
*
CG
-.257*
HCG
-.145
GPA
-.085
HSGPA ECG
-.067
-.216
44
44
41
40
1
.852
44
Pearson Correlation
-.145
.852
N
44
Pearson Correlation
N
**
44
**
.579
**
41
.445
44
**
40
**
44
**
.871**
1
.502
44
44
41
40
44
-.085
.579**
.502**
1
.628**
.755**
41
41
41
41
40
41
1
.667**
Pearson Correlation
-.067
.445
N
40
40
Pearson Correlation
-.216
.853
N
44
44
**
.374
**
40
**
.871
44
.628
**
40
**
.755
41
.374
.853**
40
**
.667
40
40
**
1
44
*. Correlation is significant at the 0.05 level (1-tailed).
**. Correlation is significant at the 0.01 level (1-tailed).
CG= Course Grade ECG = Expected Course Grade HCG –– Homeowork Grade GPA = College GPA HSGPA
= High School GPA
23
Table 8
Step Regression for Academic Performance Model
Model
Change Statistics
R Square
F Change
df1
df2
Sig. F Change
Change
1
.216a
10.761
1
39
.002
2
.656
b
194.891
1
38
.000
.078
c
58.490
1
37
.000
.002
d
1.305
1
36
.261
3
4
a. Predictors: SL
b. Predictors: SL, GPA
c. Predictors: SL, GPA, HCG
d. Predictors: SL, GPA, HCG, HSGPA
CG= Course Grade ECG = Expected Course Grade HCG –– Homeowork Grade GPA = College GPA HSGPA
= High School GPA
Comparing the service learning sample group to the non-service learning sample group,
mean expected course grades are slightly higher for non-service learning students at 2.88 versus
2.4 for service learning students. For the components that make up expected course grade, the
mean scores for college grade point average and high school grade point average were slightly
higher for non-service learning students, suggesting students who were better prepared for
college work. But the largest difference between the two groups was in course grade for
homework, with the mean score at 2.8 for non-service learning students and the mean score for
service learning students at 2.2. Course grade for homework is a measure of effort in this model,
so the results suggest service learning students put somewhat less effort into the class than their
non-service learning counterparts.
24
Table 9
Academic Performance Descriptive Statistics of Non-Service Learning Students
N
Range
Minimum
Maximum
Mean
Std. Deviation
Variance
CG
25
5.05
.00
5.05
3.4788
1.36350
1.859
ECG
25
3.52
1.04
4.56
2.8808
1.02441
1.049
HCG
25
5.00
.00
5.00
2.8004
1.96029
3.843
GPA
25
3.00
1.00
4.00
2.7840
.77217
.596
HSGPA
25
2.54
1.36
3.90
2.8698
.66934
.448
Valid N (listwise)
25
CG= Course Grade ECG = Expected Course Grade HCG –– Home-work Grade GPA = College GPA HSGPA =
High School GPA
Table 10
Academic Performance Descriptive Statistics of Service Learning Students
N
Range
Minimum
Maximum
Mean
Std. Deviation
Variance
CG
19
5.12
.00
5.12
2.6437
1.85715
3.449
ECG
19
4.65
.00
4.65
2.4021
1.19004
1.416
HCG
19
5.03
.00
5.03
2.2337
1.96750
3.871
GPA
16
2.93
1.07
4.00
2.6500
.80068
.641
HSGPA
15
2.29
1.71
4.00
2.7707
.82498
.681
Valid N (listwise)
15
CG= Course Grade ECG = Expected Course Grade HCG –– Home-work Grade GPA = College GPA HSGPA =
High School GPA
DISCUSSION
The unexpected result of this study was that the null hypothesis was supported for all
three research questions. This study found there is no relationship between service-learning and
improved levels of personal set characteristics. Students enrolled in service-learning courses
showed a decrease in two of the four personal characteristics: openness and conscientiousness,
although the differences were not statistically significant. Parker-Gwin & Mabry, 1998 reported a
25
similar finding when service-learning is required rather than optional and when time for
reflection is limited or not included in the course. The current study did not control for
characteristics of the service learning experience, so it is not known whether service learning was
required or optional. Previous research unrelated to service learning has found that personal set
characteristics are relatively set and unchanging (Costa & McCrae, 1994). This would explain
the relatively small changes between the pre-test and post-test in this study.
Comparison of pre and post survey results for all students found statistically significant
changes in two of the empathy questions and one of the self-efficacy questions (Table 3). This
could be explained by students’’ exposure to the college experience and the general maturing
process during the survey period, but it contradicts the findings of Costa and McCrae (1994) that
personal characteristics are relatively set and unchanging. These changes were seen in both
service-learning students and non-service-learning students, ruling out a relationship to service
learning.
This study found no relationship between service-learning and improved levels of civic
engagement. Non-service learning students showed greater increases in civic engagement than
their service-learning counterparts, but the differences were small and not statistically significant.
These findings mirror results reported by Hudson (1996), Miller (1994), and Parker-Gwin and
Mabry (1998). The Parker-Gwin and Mabry study in particular used a research design and
sample size similar to this study.
Comparison of pre- and post-service results for all students found significant differences
in two of the civic awareness questions and one of the civic-efficacy question (Table 6). This
could be explained by students’’ exposure to the college experience and the general maturing
26
process during the survey period. These changes were seen in both service-learning students
and non-service-learning students, ruling out a relationship to service learning.
With regard to academic performance, the statistically significant negative relationship
between service learning and improved academic performance found in this study was an
unexpected result. Particularly puzzling was the finding that course grades were lower in service
learning classes in large part because students put less effort into the class as measured by course
homework grade. This contradicts extensive previous research that found a positive relationship
between service learning and improved academic performance (Astin & Sax, 1998, Tartter, 1996,
Vogelgesang & Astin, 2000) or found service learning to have no impact (Boss, 1994, Hudson,
1996, Parker-Gwin & Mabry, 1998).
This study did not collect quantitative data from students related directly to their service
experience. However Parker-Gwin and Mabry (1998) found service learning to have no impact
on academic performance, and suggested explanations based on comments in their quantitative
data. Comments from students described a general disappointment with what their service was
able to accomplish. They had come into the course expecting to do good, to meet the needs of the
less fortunate, and to feel good about doing so. When met with the realities of service and the
complexity of problems faced, students’’ comments suggested they had become discouraged. A
similar phenomenon could explain why students in service learning classes scored lower on
course homework grades than their non-service learning counterparts. The homework in service
learning classes typically relates to the service, and if students had become discouraged or
disillusioned with the service experience they may put less effort into the homework. More study
is needed in this area to test that hypothesis.
27
It should also be noted that this study defined improved academic performance strictly in
terms of course grade. Other studies that found service learning had a positive impact on
academic outcomes (Astin & Sax, 1998, Tartter, 1996, Vogelgesang & Astin, 2000) tended to
rely on students’’ assessment of their skills in critical thinking and problem analysis, and their
ability to apply what they have learned in the ““real world””. Such outcomes could be present
without a change in the course grade for the current course. This study did not measure these
outcomes. Further research is needed to explore that possibility.
Future Research
The findings of this study suggest that not all service learning experiences are created
equal. While an abundance of research has found that service learning is related to improved
personal set characteristics, improved academic outcomes, and increased civic responsibility
(Astin & Sax, 1998; Giles & Eyler, 1994: Tartter, 1996; Vogelgesang & Astin, 2000), equally
credible studies have found service learning has no effect in these areas. (Boss, 1994; Hudson,
1996; Parker-Gwin & Mabry, 1998; Miller, 1994). The difference lies in the nuances of the
service experience and this is an area that requires further study. How does required service
learning affect these outcomes compared to optional service learning? Does service learning
show greater effect on these outcomes when students are in direct contact with the clients
served? When there is direct application of concepts gleaned from service to course concepts in
the classroom, what is the effect on academic outcomes?
If previous research found instances where service learning enhances personal set
characteristics development, academic performance, and civic responsibility, this study adds to
the body of knowledge by suggesting there are instances when the reverse is true, and where
service learning may actually impede this natural development. This is an important area for
28
future study. What is the risk of service disillusionment, where students expect to ““save the
world”” and become disheartened in the face of the real life service experience? How can this
risk be mitigated through the design of the service experience? Are there academic situations
where due to the nature of the course, the nature of the students, or the two combined, service
learning is not an appropriate pedagogy?
Finally, this study was developed with the expectation that service learning enhances
personal set characteristics and that enhanced personal set characteristics would enhance
academic performance and civic responsibility. Does any causal relationship run the other
direction? Does improved academic performance result in enhanced personal set characteristics
and increased civic responsibility? Does increased civic responsibility enhance both personal set
characteristics and improved academic performance? All these questions are interesting
questions that should be further investigated to better understand the relatiobship between service
learning, personal skill-set characteristics, civic engagement and academic performance.
29
References
Asendorph, J. B., & Van Aken, M. A. G. (2003). Personality-relationship transaction in
adolescence: Core versus surface personality characteristics. Journal of Personality, 71
(4) 629-662.
Aslam, R. Jaffery, T., & Zaidi, Z. (2011). Service-learning: Increasing civic responsibility in
Pakistani students. Blackwell Medical Education, 45(5), 508-535.
Astin. A. W. & Sax. J. L. (1998). How undergraduates are affected by service participation.
Journal of College Student Development. 39(3), 251-263.
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory.
Englewood Cliffs, N. J.: Prentice Hall.
Battistoni, M. R. (2002). ““What is Good Citizenship? Conceptual Contributions from other
Disciplines”” Campus Compact: Introduction to the Service-Learning Toolkit: Reading
and Resources for Faculty, 2nd Edition, 179-186.
Barber, B. (1998). Passion for democracy: American essays. Princeton, NJ: Princeton University
Press.
Berkas, T. (1997). Strategic review of the W. K. Kellogg Foundation’s service-learning projects,
1990-1996. Battle Creek, MI: W. K. Kellogg Foundation.
Bissell, K., & Collins, S. J. (2001). Early predictors of ability in a news writing course.
Journalism & Mass Communication Educator,56 (2), 69-80.
Blickle, G. (1996). Personality traits, learning strategies, and performance. European Journal of
Personality, 10, 337––352.
30
Bonner, T., & Aspy, D. N. (1984). A study of the relationship between student empathy and GPA.
Journal Of Humanistic Counseling, Education & Development, 22(4), 149-154.
Retrieved January 14, 2012, from Ebscohost Database.
Borde, S..F. (1998). Predictors of student academic performance in the introductory marketing
course, Journal of Education for Business, 75(5), 302-306.
Boss, J.A. (1994). The effect of community service on the moral development of college ethics
students. Journal of Moral Development. 23(2), 183-198.
Bouffard-Bouchard, T. (1990). Influence of self-efficacy on performance in a cognitive task.
Journal of Social Psychology, 130(3), 353-363.
Chamorro-Premuzic, T., & Furnham, A. (2003). Personality predicts academic performance:
Evidence from two longitudinal university samples. Journal of Research in Personality,
37 (4), 319––338.
Costa, P. T., Jr., & McCrae, R. R. (1992). NEO PI-R professional manual. Odessa, FL:
Psychological Assessment Resources, Inc
D’’Souza, K. A. & Maheshwari, S. K. (2010). Factors influencing student performance in the
introductory management science course, Academy of Educational Leadership Journal.
14(3), 99-119.
Degelman, C. (2000). Fostering civic responsibility through service-learning. Service-learning
Network, 8(1), 1-17
De Raad, B., & Schouwenburg, H. C. (1996). Personality in learning and education: A review.
European Journal of Personality, 10(5), 303––336.
Digman, J. M. (1989). Five robust trait dimensions: Development, stability, and utility. Journal
of Personality, 57 (2), 195––214.
31
Diseth, Å. (2003). Personality and approaches to learning as predictors of academic achievement.
European Journal of Personality, 17(2), 143-155.
Duff, A., Boyle, E., Dunleavy, K., & Ferguson, J. (2004). The relationship between personality,
approach to learning and academic performance. Personality and Individual Differences,
36 (8), 1907––1920.
Eskew, R.K. & Faley, R.H. (1988). Some determinants of student performance in the first
college-level financial accounting course, The Accounting Review, 63(1), 137-147.
Eyler, J., Giles Jr, D. E., & Braxton, J. (1997). The impact of service-learning on college
students. Michigan Journal of Community Service Learning, 4, 5-15.
Eyler, J., & Giles Jr, D. E. (1999). Where's the Learning in Service-Learning? Jossey-Bass
Higher and Adult Education Series. Jossey-Bass, Inc., 350 Sansome St., San Francisco,
CA 94104.
Eyler, J., Root, S., & Giles Jr, D. E. (1998). Service-learning and the development of expert
citizens: Service-learning and cognitive science. In R.G. Bringle. & D. K. Duffy (Eds.),
With service in mind: Concepts and Models for Service-learning in psychology, (pp. 85––
100).
Furco, A., Muller, P., & Ammon, M. S. (1998). The civic responsibility survey. Berkeley, CA:
Service-Learning Research & Development Cent.
Gottlieb, K., & Robinson, G. (2004). Integrating civic responsibility into the curriculum.
Community College Journal-Washington DC, 74(3), 25-26.
Gray, M.J., Ondaatje, E.H., Fricker, R., Geschwind, S., Goldman, C.A., Kaganoff, T., Robyn, A.,
Sundr, M., Vogelgesang, L., & Klein, S.P. (1998). Coupling Service and Learning in
32
Higher Education: The Final Report of the Evaluation of the Learn and Serve America,
Higher Education Program. The RAND Corporation.
Hamilton, S. F., & Zeldin, R. S. (1987). Learning civics in the community. Curriculum
Inquiry,17, 407-420.
Hudson, W. E. (1995). Combining Community Service and the Study of American Public Policy.
Paper presented at the Annual Meeting of American Political Science Association (Chicago, IL,
August 30-September 3, 1995).
Hunter, S., & Brisbin A. R. (2000). The impact of service-learning on democratic and civic
values. Political Science and Politics, 33(3), 623-626.
Kalbers, L. P. & Weinstein, G. P. (1999). Student performance in introductory accounting: A
multi-sample, multi-model analysis, Accounting Educators’ Journal. Retrieved Month 0,
2012, from http://www.aejournal.com/ojs/index.php/aej/article/view/11.
Kendrick, J.R. (1996). Outcomes of service-learning in an introduction to sociology course,
Michigan Journal of Community Service Learning. 3(1), 72-81.
Komarraju, M., & Karau, S. J. (2005). The relationship between the big five personality traits
and academic motivation. Personality and Individual Differences, 39 (3), 557––567.
Kruck, S.E. & Lending, D. (2003). Predicting academic performance in an introductory collegelevel IS course. Information Technology, Learning, and Performance Journal, 21(2), 915.
Mabry, J. B. (1998). Pedagogical variations in service-learning and student outcomes: How time,
contact, and reflection matter. Michigan Journal of Community Service Learning, 5 (1),
32-47.
33
Marsh, H. W., Trautwein, U., Ludtke, O., Koller, O., & Baumert, J. (2006). Integration of
multidimensional self-concept and core personality constructs: Construct validation and
relations to well-being and achievement. Journal of Personality, 74 (2), 403––456.
Martin, J. H., Montgomery, R. L., & Saphian, D. (2006). Personality, achievement test scores,
and high school percentile as predictors of academic performance across four years of
coursework. Journal of Research in Personality, 40 (4), 424––431.
McCrae, R. R., & John, O. P. (1992). An introduction to the five-factor model and its
applications. Journal of Personality, 60(2), 175-215. doi:10.1111/j.14676494.1992.tb00970.
Mehrabian, A., & Epstein, N. (1972). A measure of emotional empathy. Journal of Personality,
40 (4), 525-543.
Melchior, A. (1999). Summary report: National evaluation of Learn and Serve America.
Waltham, MA: Center for Human Resources, Brandeis University.
Miller, J. (1994). Linking traditional and service-learning courses: outcome evaluation utilizing
two pedagogically distinct models, Michigan Journal of Community Service Learning.
1(1), 29-36.
Parker-Gwin, R., & Mabry, J. B. (1998). Service learning as pedagogy and civic education:
Comparing outcomes for three models. Teaching Sociology, 26(4), 276-291.
Paunonen S. V., & Ashton, M. C. (2001). Big five predictors of academic achievement. Journal
of Research in Personality, 35 (1), 78––90.
Pavey, L., Greitemeyer, T., & Sparks, P. (2012). ““I Help Because I Want to, Not Because You Tell
Me to””: Empathy increases autonomously motivated helping. Personality & Social
Psychology Bulletin, 38(5), 681-689.
34
Perry, L. J., & Katula C. M. (2001). Does service affect citizenship? Administration and Society,
33(3), 330-365.
Piedmont, R. L. (1998). The revised NEO Personality Inventory: Clinical and research
applications. New York: Plenum.
Pintrich, P. R., & Schunk, D. H. (2002). The role of goals and goal orientation. In P. R. Pintrich
& D. H. Schunk (Eds.), Motivation in education: Theory, research and application (2nd
ed., pp. 190––242). Englewood Cliffs, NJ: Simon & Schuster.
Poropat, A. E. (2009). A meta-analysis of the five-factor model of personality and academic
performance. Psychological Bulletin, 135(2), 322––338.
Rammstedt, B. & John, O. P. (2007). Measuring personality in one minute or less: A 10item short version of the Big Five Inventory in English and German. Journal of
Research in Personality, 41 (1), 203-212.
Seifert, T. L., & O’’Keefe, B. A. (2001). The relationship of work avoidance and learning goals to
perceived competence, externality and meaning. British Journal of Educational
Psychology, 71 (1), 81––92.
Shell, D. F., Colvin, C, & Bruning, R. H. (1995). Self-efficacy, attributions, and outcome
expectancy mechanisms in reading and writing achievement: Grade-Level and
achievement-level differences. Journal of Educational Psychology, 87, 386-398.
doi:10.1037/0022-0663.87.3.386.
Solberg, V. S., O’’Brien, K., Villareal, P., Kennel, R., & Davis, B. (1993). Self-efficacy and
Hispanic college students: Validation of the College Self-Efficacy Instrument. Hispanic
Journal of Behavioral Sciences, 15(1), 80-95. Retrieved December 10, 2011, from the
Ebscohost Database.
35
Spreng, R. N., McKinnon, M. C., Mar, R. A., & Levine, B. (2009). The Toronto Empathy
Questionnaire: Scale development and initial validation of a factor-analytic solution to
multiple empathy measures. Journal of Personality Assessment, 91 (1), 62-71.
Stroessner, S. J., Beckerman, L., & Whittaker, A. (2009). All the world's a stage? Consequences
of a role-playing pedagogy on psychological factors and writing and rhetorical skill in
college undergraduates. Journal Of Educational Psychology, 101(3), 605-620.
doi:10.1037/a0015055
Tartter, V.C. (1996). City College Report to FIPSE. New York: City College Research
Foundation.
Tempelaar, D. T., Gijselaers, W. H., Schim Van Der Loeff, S., & Nijhuis, J. (2007). A structural
equation model analyzing the relationship of student achievement motivations and
personality factors in a range of academic subject-matter areas. Contemporary
Educational Psychology, 32 (1), 105––131.
Vogelgesang, L.J. & Astin, A.W. (2000). Comparing the effects of service-learning and
community service, Michigan Journal of Community Service Learning. 7(1), 25-34.
Walzer, M. (1970). Obligations: essays on disobedience, war and citizenship. Cambridge, MA:
Harvard University Press.
Williams, R. D. (2000). Participants in, not spectators to democracy: The discourse on civic
responsibility in higher education. Michigan Journal of Community Service
Learning,7(1) , 158-164.
Youniss, J., McLellan, J. A., & Yates, M. (1997). What we know about engendering civic
identity. American Behavioral Scientist, 40 (5), 620––631.