A COMPARISON OF MOTIVATION OF AN EARLY ALERT SYSTEM IN K-12 AND
HIGHER EDUCATION
by
Kathryn Camilleri
A Senior Honors Project Presented to the
Honors College
East Carolina University
In Partial Fulfillment of the
Requirements for
Graduation with Honors
by
Kathryn Camilleri
Greenville, NC
May 2015
Approved by:
Dr. Ricky T. Castles
Department of Engineering, College of Engineering and Technology
Dr. Linda L. Mellish
Associate Director, Assessment and Research in Campus Living and the Office of Student
Transitions
Abstract
The effects of early alert systems on student motivation in K-12 and higher education are
examined throughout this paper. The two early alert systems studied are Starfish™, which is
designed to inform students of academic progress, connect students with appropriate campus
resources, and support student success and retention and ClassDojo, which is an immediate
behavioral management system and skill tracking tool designed for use with elementary grade
students. Early alert systems are used to make students aware of their current progress or
standing and encourage them to alter their actions to be more successful. This paper investigated
the behavioral actions of students after receiving the notifications from each early alert system.
Data was gathered and studied to determine the behavior modifications, if any, after the use of
the systems. Based on this study, it was determined that the same motivation strategies do not
work for all students. Some students went to tutoring based on notifications from teachers, while
some did not. Likewise, some students changed to more appropriate behavior based on their
ClassDojo points, while others did not.
Table of Contents
Introduction ..................................................................................................................................... 4
Literature Review............................................................................................................................ 6
Methodology ................................................................................................................................. 10
Data Analysis ................................................................................................................................ 11
Limitations .................................................................................................................................... 17
Conclusion .................................................................................................................................... 17
Introduction
This study examines the effects of early alert warning systems on student academic
performance and contrasts the outcomes of the use of an early alert warning system at a
university with a similar system used with elementary age students. East Carolina University
currently uses an early alert system called Starfish™, which is designed to inform students of
academic progress, connect students with appropriate campus resources, and support student
success and retention (Starfish Retention Solutions | Start to Finish, 1998). Starfish™ allows
instructors to give praise (kudos), or raise concerns (flags) about a student’s academic
performance within a specific course.
ClassDojo is an immediate behavioral management system and skill tracking tool
designed for use with elementary grade students. ClassDojo is an application that can be
downloaded to any computer, iPad or iPhone. Through ClassDojo, a student can gain or lose
feedback points based on their social behavior. ClassDojo allows the user to create specific
points for certain behaviors in the classroom. ClassDojo helps improve student behavior and
build strong learning habits by awarding students specific feedback points for demonstrating
positive behavior in the classroom. Students either receive “positive” points or lose “needs work”
points. “Positive” points are worth one point and “needs work” points are worth negative one
point. ClassDojo “reinforces positive behaviors and skills by giving feedback points and easily
engages parents with ClassDojo Messaging” (ClassDojo, 2011).
Starfish™ provides students with intangible rewards. The students who receive a flag or
kudo from Starfish™ are at their own will to benefit from the notification. ClassDojo provides
the students with tangible rewards. After the students receive a certain number of points, they
will obtain a prize. With ClassDojo, the students are being rewarded with a physical item to
show for their success in the classroom.
ClassDojo is a system that is based on monitoring the student’s social behavior rather
than their academic behavior. The students must exhibit positive learned behaviors in order to
receive a point. Starfish™ is based on academic behavior. The students receive a notification
based on their academic performance in a specific class. Also, ClassDojo gives the students
immediate feedback about their behavior. If they are performing a specific behavior, appropriate
or not, they will know right away. Starfish™ notifications are only sent to students when faculty
members feel the need to raise alerts.
There is also a difference between the student support networks of the two systems.
ClassDojo’s network consists of the teacher and the student’s parents or guardians at home. It is
not a very large network and there are no other avenues of resources. With ClassDojo, there is a
type of peer pressure from the class also, since the whole class can always see the number of
points anyone has and when points get added or taken away. Starfish™ has a larger network.
When faculty members give a student a flag, he/she notifies other networks of support, such as
an advisor and the on-campus tutoring center. Certain special populations of students have
additional levels of support in their network such as athletes, members of Greek organizations,
and students in the Honors College. Since, this system connects various resources for support,
students are provided with multiple options from which they can choose which one will be most
beneficial to them.
These two systems are alike because they both provide the students with knowledge
about their behaviors. Both systems offer praise and disapproval in a way that the students can
visualize. Good behavior is being rewarded and bad behavior is being addressed in both systems.
Also, by warning students about their progress, Starfish™ and ClassDojo both have the
capability to affect a student to where he/she will modify his/her behavior.
Literature Review
Many schools use an Early Alert Warning System (EAS) or some type of behavior
modification system. An EAS would be most apparent in higher education whereas a behavior
modification system would be utilized in K-12. A second grade teacher, Janet Brooks, uses
ClassDojo. Mrs. Brooks was interviewed and asked why she chose to use ClassDojo. She said
she originally chose it because it was a free application and it was simple and easy to use. She
liked it because it can be used with an iPad, iPhone and computer. She was asked if she had
always used ClassDojo. Her response was no, but she has since she got an iPad, which was two
years ago. When asked what behavior management system she used before, she said that she has
tried a few different ones. She tried a clip chart where the students move their binder clip up on
the chart for good behavior and move their clip down for bad behavior. She also used a bubble
gum machine cut-out where the students would get stickers to put onto the bubble gum machine.
After ten stickers, the students would receive a prize. Mrs. Brooks said she went to ClassDojo
because it was individualized. She could give individual students points without rewarding or
punishing the whole class. She also chose ClassDojo because she can see the data’s overall
percentage. Each week, ClassDojo shows students overall percentage of good behavior versus
bad behavior in a pie chart. Last, Mrs. Brooks was asked if she had observed this system
behaviorally. She said yes and that the children love it because they can see the points on the
screen. She has seen them change their behavior according to the ClassDojo points. She has also
noticed that other teachers use the same system in the school which helps the kids recognize its
significance. Her favorite part of ClassDojo is the technology aspect. “I love the technology
component of it. It’s simple, easy and convenient.” Mrs. Brooks also said that at the beginning of
the year, she uses this system for everything. Every little action that the students do, she rewards
them with points. “I want to get that good taste in their mouth of how it feels to get points.” As
the year goes on, she uses ClassDojo less and less. She wants students to work for their points.
She started by giving the students a prize after ten points, and now she gives them a prize after
twelve. She says she will increase that number to about fifteen or seventeen by the time the year
is over. She has a poster where the students put a sticker after they receive the required number
of points. The poster tells the students what prize they receive each time.
Based on this one teacher’s experience, ClassDojo is helpful because it demonstrates to
students what the teacher’s expectations are early and often. “Teaching behavioral expectations
and rewarding students for following them is a much more positive approach than waiting for
misbehavior to occur before responding” (Yeung, et al., 2009). William Glasser’s model (1992)
of education promotes that students take responsibility for making their own choices in behavior.
Early alert warning systems and behavior modification systems, such as Starfish™ and
ClassDojo, hold students accountable for their actions.
Yeung, Mooney, Barker and Dobia studied the impacts of Positive Behaviour for
Learning (PBL) on the important psychosocial outcomes of learning. PBL is a school-wide
system in Australia based on the USA model of Positive Behavioral Interventions and Supports
(PBIS). Its goal is to teach students behaviors that are appropriate for their students in primary
schools. ClassDojo is one system in the United States that also teaches students behaviors that
are appropriate. ClassDojo’s goal is to develop traits in students that will lead to long-term
success. By using ClassDojo, a teacher is correcting inappropriate behavior early and therefore
increasing the chance that a student will demonstrate better behavior in the future. Students’
positive behavior is expected to have positive influences on their social development (Yeung, et
al., 2009). Six schools, four serving as the experimental group and two being the control group,
in the Western Sydney Region of Australia were selected to execute the PBL system in their
schools. After implementing PBL for 9 months, the data showed that the students in the PBL
schools liked going to school more than the control students. PBL also improved the student’s
perception of competence in English, the student’s relationship with their parents, and the
students planning of their schoolwork and study. All the reports showed that the PBL students
never scored lower than the control students (Yeung, et al., 2009). Furthermore, since research
has established the importance of behavior modification systems in a classroom, this study
investigated student’s motivation to succeed using a behavior modification system.
When using a behavioral modification system, it is important to connect with the parents.
ClassDojo allows parents to view their child’s progress at home. Parents can see specifically
which feedback points the student gained or lost and for what reason. Also, teachers and parents
can send messages through ClassDojo about a child’s progress. Some behavior modification
systems are used and focused at both school and home. The U.S. Department of Education
examined the effects of First Step to Success. First Step is “a school and home based program
intended to improve outcomes for students with moderate to severe behavior problems who may
be at risk for academic failure” (U.S. Department of Education, 2013). First Step was reported to
increase student academic engaged time and teacher assessment of academic competence (U.S.
Department of Education, 2013). Similarly, ClassDojo could be used at school and reinforced at
home with the help of involved parents.
The purpose of an early alert warning system is to alert students of their progress so that
they are more likely to modify behaviors necessary to succeed. EAS are more common in higher
education. Potential psychosocial predictors of freshman academic achievement and retention
have been examined in an introductory psychology and sociology classes at a private university
(DeBerard, Spielmans & Julka, 2004). The findings of this study show that there was a
statistically significant correlation between cumulative GPA and retention. Students who
perform better in school are more likely to stay in school. EAS help students become aware of
their current academic standing. Starfish™ not only makes students aware of their standing, but
it also connects the students to resources on campus to promote success. Starfish™ directs
students to tutoring centers, writing centers, and even professor’s office hours.
In any EAS or behavior modification system, communication is vital. With Starfish™,
the teachers are specifically sending out notifications to students. By doing so, the teacher is also
telling the student that they are available for communication outside of class. Using an EAS
demonstrates that teachers want to help their students succeed. Communication between teacher
and student has proven to be significantly beneficial. The frequency, length and immediacy of
out of class communication between students and faculty are all important factors. Verbal
immediacy is most closely linked to the frequency and length of out-of-class communication
(Jaasma & Koper, 1999). Teachers can provide feedback and communication through behavior
modification systems. ClassDojo provides the students with immediate feedback from the
teacher. Teachers who create a positive rapport and engage the students in conservation are
going to have better relationships with their students. Students need to be constantly aware of
their progress. Students will also benefit if they feel that the teacher cares about their success.
Motivation is another aspect that can be increased with communication. Frequency and length of
contact with professors correlates positively with student motivation (Jaasma & Koper, 1999).
Communication from teachers combined with these warning systems motivate students to
perform to their best ability. Students who receive a Starfish™ flag are encouraged to follow up
with the Pirate Tutoring Center (PTC) as a possible solution to seek help while students who
went to the PTC prior to receiving a Starfish™ kudo will exhibit motivation by continuing to
attend tutoring at the PTC. Likewise, students will be rewarded for their positive behavior and
therefore will exhibit appropriate behavior in the classroom in order to receive more ClassDojo
points.
Methodology
This study examined the effects of an EAS and a behavioral modification system on
student motivation. It investigates whether students were more motivated to go to the PTC after
receiving a flag and how many times. It also looks at whether students changed their behavior to
get more successful results in the classroom.
A sample population of undergraduate students in introductory mathematics and
psychology classes at a public university was studied. Microsoft Excel was used to choose the
data to include. The data was provided by an administrator of the Starfish™ system. The MATH
1065, College Algebra, and PSYH 1000, Introduction to Psychology, sections were pulled out of
that data set. The data contained information about the student, the section of the class, the
professor, whether the student received a flag or kudo, and the date that notifications were raised.
Then, with Excel, a random sampling of the overall data was selected. The random sample of the
data was performed in order to produce 200 participants in each class for two semesters, Fall
2013 and Spring 2014.
Then, those 200 participants were compared with another set of data showing which
students went to the PTC. The students, who both received alerts and visited the PTC, were then
used. There were 17 students in MATH 1065 in fall 2013, 15 students in PSYC 1000 in fall
2013, 20 students in MATH 1065 in spring 2014 and 16 students in PSYC 1000 in spring 2014
comprising a total of 68 students who met the selection criteria. The students who did not go to
the PTC were then eliminated from the study. The data was then combined onto one spreadsheet
and it could be observed if the students went to the PTC after receiving a kudo or flag.
The data was then coded and colored. The students would receive a green color if they
went to tutoring after receiving a flag or kudo. The students would receive a red color if they did
not go to tutoring after receiving a flag or kudo. The data was then coded based on the following
key: F- NT= flag but no tutoring, K- NT= kudo but no tutoring, T- BF= tutoring before flag, TBK= tutoring before kudo, T- AF= tutoring after flag, T- AK= tutoring after kudo.
Data Analysis
Starfish™ Results
Out of the eight hundred students analyzed, only 68 students chose to go to the PTC after
receiving an alert. There is no obvious relationship between receiving a notification and going to
tutoring. Some students go to tutoring before they receive a notification but then don't go after
they do. Some students go to tutoring after one notification but then not after the second or third.
Students would go to the PTC after both flags and kudos, but more students went after receiving
flags. The students, who do go to tutoring after receiving notifications, also go to tutoring
different amounts of times. No relationship can be concluded; there is no conclusive pattern.
Table 1 Number of students who attended PTC after Flag
Class
Fall 2013
Spring 2014
MATH 1065
8
14
PSYC 1000
9
10
Table 2 Number of students who attended PTC before flag
Class
Fall 2013
Spring 2014
MATH 1065
6
6
PSYC 1000
3
0
For the MATH 1065 class, 59% of students went to the PTC after receiving a flag. Sixtyone percent of students in the PSYC 1000 class went to the PTC after receiving a flag. Only 32%
of students in MATH 1065 went to the PTC before receiving a flag while 10% of students in
PSYC 1000 went before receiving a flag.
Danielle is a student who took PSYC 1000 in Spring 2014. She received a flag during the
first month of the semester. Three days later, Danielle visited the PTC for help in PSYC 1000.
She continued to visit the PTC 13 more times that semester. In this situation, Danielle was
motivated to visit the PTC after she received a flag. The fact that she went fourteen times
demonstrates her motivation to succeed in the class.
Kristen is a student who took PSYC 1000 in Fall 2013. Kristen went to the PTC before
she ever received a flag from Starfish™. A few weeks later when she received a flag, Kristen
never went to tutoring again. In this case, the notification of a flag did not motivate Kristen to go
back to the PTC center.
Table 3 shows the number of students who went to the PTC before a flag, but did not go
after receiving a flag. Nine of the sixty-eight students were not motivated to go to the PTC after
receiving a flag, even though they had gone previously.
Table 3 Number of students who went to PTC before flag but not after
Class
Fall 2013
Spring 2014
MATH 1065
5
2
PSYC 1000
2
0
ClassDojo Results
Sawyer is an elementary child in a classroom that uses ClassDojo. Sawyer’s behavior has
improved because of ClassDojo. In the first month of school, Sawyer lost eight points for talking
in class. After losing those eight points, he has modified his behavior by talking less during class.
His behavior in the classroom has been notably different. He is talking less and on task more
frequently. Since late September, Sawyer has only lost two points for talking and has received
six points for being on task. This is compared to the four points he received for being on task and
eight points he lost for talking during the first month of school, shown in Table 3. Since the
beginning of the year, Sawyer has a 79% positive rate of behavior. He has received 61 “positive”
points and lost 16 “needs work” points, over half of them being in the first month of school.
Figure 1 shows Sawyer’s overall percentage of ClassDojo points and the specific points received.
Figure 1 Sawyer's overall ClassDojo points
All of the “needs work” points that Sawyer has received are shown in Table 4. This table
shows the date that the point was given and the reason.
Table 4 Sawyer's "needs work" points
Date
Reason given
August 29th
Talking
September 2nd
Talking
September 3rd
Talking
September 3rd
Talking
September 8th
Talking
September 9th
Issues in encore class
September 15th
Talking
September 16th
Talking
September 22nd
Talking
October 2nd
Talking
October 15th
October 23rd
Not following substitute’s directions/continuous talking
even after warnings
Not following substitute’s directions/continuous talking
even after warnings
Poor manners in the lunchroom
November 12th
Talking
November 21st
Poor manners in the lunchroom
November 24th
Playing in the bathroom
October 15th
Figure 2 shows how many rewards Sawyer (student #18) has received for ClassDojo
points compared to the rest of the students in his class. This chart is based on the entire academic
year thus far. This specific teacher allows the students to receive a sticker on the chart when they
accumulate twelve ClassDojo points (she started the year only requiring ten points). Sawyer has
three stickers on the chart, which corresponds to about thirty-five ClassDojo points. This chart
also shows what the students receive for each reward.
Figure 2 Rewards Chart
Figure 3 is the visual of ClassDojo that the students are able to see throughout the day.
This shows how many ClassDojo points the students currently have compared to the rest of the
class. Each student has their own avatar with a small figure and their name. The number in the
green or red circle is the number of point that the student has at that moment. The teacher clicks
on the name and icon of the students to give and take away points.
Figure 3 ClassDojo Home Screen
While some students modify their behavior in the classroom to receive more ClassDojo
points, others do not. Jaterria continues to lose ClassDojo points throughout the day for actions
such as, talking, playing in the bathroom, and being unkind to classmates. However, she has
continued this behavior on a daily basis. This behavior modification system is not motivating
Jaterria enough to change her behavior.
Limitations
The limitations of this study include that it only looked at one action that was taken by
the university students, going to the PTC. Students could have gone to office hours, emailed a
professor, met with their advisor, or studied more. There were also other services available, such
as the Math Cave, that were not taken into consideration. The Math Cave is a computer lab that
offers educational modules for College Algebra instruction. This study does not look at the other
resources or actions that the students could have utilized.
Conclusion
While some students are motivated by alerts to change their behavior, others are not.
Based on the Starfish™ data, some students go to tutoring before receiving a flag, some go to
tutoring after, and others did not do either. For some students, this type of motivation does not
work. The same can be said for ClassDojo. Some students, like Sawyer, change their behavior to
be more successful. While other students, like Jaterria, are not motivated to do so. The same
motivation tactics does not work for all students. While such systems are not successful in
modifying all student behaviors, there are enough success stories involving positive behavior
change that the use of such systems is justified.
Bibliography/References
ClassDojo. (2011). Retrieved November 5, 2014, from https://www.classdojo.com/resources
DeBerard, M. S., Spielmans, G., & Julka, D. (2004). Predictors of academic achievement and
retention among college freshmen: A longitudinal study. College Student Journal, 38(1),
66-80.
Jaasma, M.A. & Koper, R.J. (1999). The relationship of student-faculty out-of-class
communication to instructor immediacy and trust and to student motivation.
Communication Education. 48(1) 41-47.
Starfish Retention Solutions | Start to Finish. (1998, January 1). Retrieved November 1, 2014,
from http://www.starfishsolutions.com/
U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse.
(2013, September). WWC review of the report: Assessing the effectiveness of First Step
to Success: Are short-term results the first step to long-term behavioral improvements?
Retrieved from http://whatworks.ed.gov
Yeung, A. S., Mooney, M., Barker, K., & Dobia, B. (2009). Does School-Wide Positive
Behaviour System Improve Learning in Primary Schools? Some Preliminary Findings.
New Horizons In Education, 57(1), 17-32.