ProgramEvaluationSummary
July2011thoughNovember2012
ToevaluateworkshopeffectivenessofthePREPaREcurriculum,collectionofparticipant
satisfactionandpre‐andpost‐workshopdataisastandardelementofallworkshop
offerings.ThisdocumentsummarizestheprogramevaluationdatacollectedfromJuly2011
thoughNovember2012.Thesedataprovidecontinuedinsightintoparticipantworkshop
satisfactionandprovideguidanceregardingtheextenttowhichthePREPaREcurriculum
influencesparticipantschoolcrisispreventionandinterventionattitudesandknowledge.
WORKSHOP1
Ofthepossible1021responsestherewere956responsestothepretestand/orposttest
(65responsestotheevaluationonly).Atotalof724(75.7%)participantscompletedvalid
pre‐andpost‐testsand635(66.4%)participantscompletedvalidevaluationsurveys.
DemographicInformation
Table1offersdescriptivestatisticsofparticipantdemographicinformation.The
distributionofyearsintheparticipants’currentprofessionswas:0years(20.9%;n=213),
1‐5years(27.4%;n=280),6‐10years(16.5%;n=168),11ormoreyears(24.3%;n=248),
missing(11.0%,n=112).ThenumberofpriorschoolcrisistraininghoursWorkshop1
participantsreportedvariedwiththemajorityofparticipantsindicatingtheyeitherhad
morethantenhours(35.6%,n=363)or0hours(25.4%,n=259);oftheremaining
participants11.9%(n=121)had1‐5hoursofpriortraining,15.7%(n=160)had6‐10
hours,and118responses(11.6%)weremissing.
Table1.DemographicDataofParticipantsfortheCrisisPreventionandPreparedness
Workshop(Workshop#1;N=1021).
Workshop#1
Demographics
N
Percent
Occupation
MentalHealthProfessionals
600
58.8
SchoolPsychologist
425
41.6
SchoolSocialWorker
31
3.0
AgencySocialWorker
2
0.2
SchoolCounselor
119
11.7
OtherSchoolBasedMentalHealth
11
1.1
OtherCommunityBasedMentalHealth
12
1.2
Educators
197
19.3
GeneralEducationTeacher
16
1.6
SpecialEducationTeacher
8
0.8
SchoolAdministrator
124
12.1
DistrictAdministrator
33
3.2
UniversityProfessor
16
1.6
Health‐Care
22
2.2
SafetyOfficer
54
5.3
Other
26
2.5
Missing
122
11.9
GraduateStudent
278
27.2
Intern(SchoolPsychology)
98
9.6
Race/Ethnicity
Asian
20
2.0
BlackorAfricanAmerican
126
12.3
White
673
65.9
Hispanic/Latino
43
4.2
Other(AmericanIndian,PacificIslander,Other)
23
2.3
Missing
136
13.3
WorkshopSatisfaction
Oftheavailable751WorkshopEvaluationforms,635includedcompleteinformation.
Overall,participantsreportedahighdegreeofsatisfactionwiththeirworkshopexperience
(M=4.5outof5;SD=.48).Participantsappearedtoreportsimilarsatisfactionforboth
workshopexperience(M=4.63,SD=.50;Questions1,12‐20)andworkshopobjectives
(Questions2‐11;M=4.38,SD=.58).Specificquestionsandparticipantresponsesare
summarizedinTable2.
Table2.ParticipantWorkshopSatisfaction.ParticipantRatingsfortheCrisis
PreventionandPreparednessWorkshop(Workshop#1).
Workshop#1
Question
N
Mean
SD
CrisisTeam
1.Theobjectiveswereclearlystated.
730
4.70
.67
TheworkshopobjectiveswereclearlymetandIcannowidentifythe:
2.Fourcharacteristicsofacrisisevent.
732
4.39
.68
3.KeyconceptsassociatedwiththePREPaRE
733
4.46
.68
acronym.
4.Fourphasesofcrisismanagement.
732
4.36
.74
5.Threeconceptsrelatedtocrimeprevention
732
4.52
.68
throughenvironmentaldesign.
6.Fourelementsrelatedtopromoting
730
4.29
.74
psychologicalsafety.
7.Purposeofacomprehensivesafetyteam.
729
4.48
.68
8.MajorfunctionsoftheIncidentCommandSystem
731
4.37
.73
(ICS).
9.Guidingprinciplesincrisisplandevelopment.
10.Differencebetweenthecrisisteamresponse
planandtheschoolstaffresponseplan.
11.Threestrategiesforexaminingeffectivenessof
crisispreventionandpreparedness.
WorkshopSatisfaction
12.Thecontentwasclearandunderstandable.
13.Workshopmaterialswerewellorganized.
14.Thetrainer(s)was/werewellorganized.
15.Workshopmaterialsfacilitatedparticipation
amongparticipants.
16.Thetrainer(s)facilitatedparticipationamong
participants.
17.Thisworkshopincreasedmyknowledge.
18.Iwillbeabletoapplytheinformation/skills
learnedtomyprofessionalduties.
19.Irecommendthisworkshop.
20.Irecommendthis/thesetrainer(s).
729
729
4.33
4.30
.69
.76
713
4.36
.73
678
678
679
677
4.59
4.68
4.68
4.51
.62
.60
.63
.70
671
4.54
.67
670
669
4.64
4.53
.59
.65
668
664
4.61
4.67
.64
.63
*Allitemsona1‐5scale,with1meaningstronglydisagreeand5meaningstronglyagree.
WorkshopEffectonParticipants’AttitudesTowardSchoolCrisisWork
Table3offersdescriptivestatisticsforthepre‐andpost‐workshopquestionsaskedof
participantstoassesstheirattitudestowardcrisispreventionandpreparedness.Despite
participantsstartingoffwithapositiveoverallmeanattitudetowardcrisisintervention
andpreventionwork,scoresstillincreasedsignificantly(becamemorefavorable,t(714)
=27.57,p=.000,etasquared=.52)fromthepre‐test(M=3.32outof5;SD=.55)tothepost‐
test(M=3.87;SD=.53).Further,whenexaminingindividualitems,participantsreporting
significantchanges(p<.05)forthebetteronallfourquestions.Thelargestgaininattitudes
towardcrisispreventionwasseenforitemone,whichindicatesparticipantsfeel
significantlymoreknowledgeableaboutthistopicafterparticipatinginthisworkshop
(t(717)=34.47,p=.000,etasquared=.62).
Anexplorationoftheassociationofdemographicfactorswithchangesinattitudefound
moderate,significantdifferencesbetweenparticipantsinattitudetowardcrisis
interventionandpreventionasafunctionofyearsspentintheircurrentprofession
(F(3,707)=10.11,p=.000,etasquared=.04),withthosewithzeroyearsofexperience
(M=.72,SD=.46)makingsignificantlylargergainsinattitudethanallotherparticipants(1‐5
yearsM=.53,SD=.51,6‐10yearsM=.51,SD=.58,11ormoreyearsM=.43,SD=.56).Thiswas
furtherexplainedbythemoderate,butsignificantdifferencefoundbetweenstudentsand
workingprofessionals,(t(684)=5.98,p=.000,etasquared=.05)whichindicatedstudents
reportedmorepositivechangeinattitudetowardcrisisprevention.Further,thenumberof
previousschoolcrisistraininghours(F(3,706)=.11.32,p=.000,etasquared=.05)was
moderatelyrelatedtochangesinattitude,withthosewithfewerhoursofpriortraining
experiencingmorepositivechanges.Lastly,therewerealsomoderate,significant
differencesbetweenparticipantsreportingdifferentoccupations(F(4,694)=8.916,p=.000,
etasquared=.05).Specifically,mentalhealthprofessionals(M=.61,SD=.50)experienced
morepositivechangeinattitudethanbotheducators(M=.44,SD=.57)andsafetyofficers
(M=.18,SD=.57),whilethosereporting“other”professionsalsoexperiencedmorepositive
changeinattitudethansafetyofficers(M=.66,SD=.53).
Table3.Workshop#1Participants’AttitudesTowardPreventionandPreparedness*
Pretest
Posttest
Question
N
Mean SD Mean
SD
1.Howknowledgeableareyouaboutschoolcrisis 322 2.40
.73
3.43
.70
preventionandpreparedness?
2.Howconfidentareyouinyourabilityto
322 2.76
.99
3.53
.80
collaboratewithotherstodevelopa
comprehensiveschoolcrisisresponse
managementplan?
3.Howenthusiasticareyoutocollaboratewith
322 3.49
.87
3.85
.79
otherstodevelopacomprehensiveschoolcrisis
responsemanagementplan?
4.Howimportantdoyoufeelschoolcrisis
321 4.62
.67
4.67
.62
preventionandpreparednessknowledgeand
skillsareintoday’sschools?
*Allitemsona5‐pointscale,withhigherscoresindicatingmorepositiveattitudes.
WorkshopEffectonParticipantSchoolCrisisWorkKnowledge
Workshop1participantresponsesindicatedlarge,significantincreasesinknowledge
acrossallparticipants(t(693)=54.59,p=.000,etasquared=.81)frompre‐test(M=4.83
outof10;SD=1.39)topost‐test(M=8.57outof10;SD=1.38).Therewerenosignificant
differencesfoundbetweenparticipantsinknowledgegainedofcrisisinterventionand
preventionasafunctionofyearsspentintheircurrentprofession(F(3,319)=.32,p=.811),
reportedprofessions(F(3,687)=.48,p=.695),amountofpreviousschoolcrisistraining
hours(F(3,687)=1.12,p=.341),oramountofknowledgegainedbetweengraduatestudents
andworkingprofessionals(t(664)=.62,p=.535).Further,therewerenosignificant
differencesbasedonparticipantoccupation(F(4,48.93)=.213,p=.930).
WORKSHOP2
Ofthepossible1457responsestherewere1357responsestothepretestand/orposttest
(100responsestotheevaluationonly).Duetoanerrorinpre‐testandpost‐testforms,only
496pre‐andpost‐testswerevalidforevaluation.However,1083(74.3%)participants
completedevaluationsurveys.
DemographicInformation
WholeSample.Table4offersdescriptivestatisticsofparticipantdemographic
informationforallWorkshop2participants.Forthisworkshop,thedistributionofyearsin
thecurrentprofessionwas:0years(14.8%(n=150),1‐5years(32.3%,n=328),6‐10years
(19.9%,n=202),11ormoreyears(32.4%,n=329),missing(0.5%,n=5).Themajorityofthe
participants(38.5%,n=390)hadmorethan10hoursofpriorschoolcrisisintervention
trainingprevioustotheworkshop;oftheremainingparticipants,23.2%(n=235)had0hrs
ofpriortraining,11.7%(n=119)had1‐5hours,25.7%(n=261)had6‐10hours,and0.9%
(n=9)oftheresponsesweremissing.
Table4.DemographicDataofParticipantsfortheCrisisInterventionandRecovery
Workshop(Workshop#2;N=1014).
Workshop#1
Demographics
N
Percent
Occupation
MentalHealthProfessionals
884
87.2
SchoolPsychologist
487
48.0
SchoolSocialWorker
112
11.
AgencySocialWorker
11
1.1
SchoolCounselor
215
21.2
OtherSchoolBasedMentalHealth
34
3.4
OtherCommunityBasedMentalHealth
25
2.5
Educators
77
7.6
GeneralEducationTeacher
10
1.0
SpecialEducationTeacher
8
0.8
SchoolAdministrator
17
1.7
DistrictAdministrator
27
2.7
UniversityProfessor
15
1.5
Health‐Care
14
1.4
SafetyOfficer
5
0.5
Other
15
1.5
Missing
19
1.9
GraduateStudent
203
20.0
Intern(SchoolPsychology)
91
9.0
Race/Ethnicity
Asian
23
2.3
BlackorAfricanAmerican
98
9.7
White
727
71.7
Hispanic/Latino
71
7.0
Other(AmericanIndian,PacificIslander,Other)
46
4.5
Missing
49
4.8
PreandPostSample.Table5offersdescriptivestatisticsofparticipantdemographic
informationforWorkshop2participantswhocompletedvalidpreandposttests.Forthis
workshop,thedistributionofyearsinthecurrentprofessionwas:0years(13.9%(n=69),
1‐5years(32.3%,n=160),6‐10years(21.8%,n=108),11ormoreyears(31.7%,n=157),
missing(0.4%,n=2).Themajorityoftheparticipants(40.9%,n=203)hadmorethan10
hoursofpriorschoolcrisisinterventiontrainingprevioustotheworkshop;ofthe
remainingparticipants,20.4%(n=101had0hrsofpriortraining,13.3%(n=66)had1‐5
hours,25.0%(n=124)had6‐10hours,and0.4%(n=2)oftheresponsesweremissing.
Table5.DemographicDataofParticipantsfortheCrisisInterventionandRecovery
Workshop(Workshop#2;N=496).
Workshop#1
Demographics
N
Percent
Occupation
MentalHealthProfessionals
440
88.7
SchoolPsychologist
269
54.2
SchoolSocialWorker
55
11.1
AgencySocialWorker
6
1.2
SchoolCounselor
88
17.7
OtherSchoolBasedMentalHealth
12
2.4
OtherCommunityBasedMentalHealth
10
2.0
Educators
37
7.5
GeneralEducationTeacher
3
0.6
SpecialEducationTeacher
3
0.6
SchoolAdministrator
14
2.8
DistrictAdministrator
14
2.8
UniversityProfessor
3
0.6
Health‐Care
6
1.2
SafetyOfficer
0
0.0
Other
7
1.4
Missing
6
1.2
GraduateStudent
65
13.1
Intern(SchoolPsychology)
38
7.7
Race/Ethnicity
Asian
16
3.2
BlackorAfricanAmerican
33
6.7
White
369
74.4
Hispanic/Latino
35
7.1
Other(AmericanIndian,PacificIslander,Other)
22
4.4
Missing
21
4.2
WorkshopSatisfaction
Ofthe1083WorkshopEvaluationforms,861werecomplete.Overall,itisconcludedthat
totalparticipantsatisfactionforworkshop2washigh(M=4.64outof5;SD=.46).
Participantsappearedtoreportequallyhighsatisfactionforbothworkshopexperience
(M=4.72,SD=.48;Questions1,8‐16)andworkshopobjectives(Questions2‐7;M=4.45,
SD=.62).SpecificquestionsandparticipantresponsesaresummarizedinTable6.
Table6.ParticipantWorkshopSatisfaction.ParticipantRatingsfortheCrisis
InterventionandRecoveryWorkshop.
Workshop#2
Question
N
Mean*
SD
WorkshopObjectives
1.Theobjectiveswereclearlystated.
1057
4.72
.70
TheworkshopobjectivesweremetandIamnowabletodothefollowing:
2.Reportimprovedattitudestoward,and
readinesstoprovide,schoolcrisisintervention.
3.Identifyvariablesthatdeterminethenumber
ofindividualslikelytraumatizedbyagivencrisis.
4.Identifytheschoolcrisisinterventions
specifiedbythePREPaREacronym
5.Identifyhowschoolcrisisinterventionfitsinto
themultidisciplinary(NIMS/ICS)schoolcrisis
response.
6.Statethetriagevariablesthatpredict
psychologicaltrauma.
7.Matchthedegreeofpsychologicaltraumarisk
totheappropriateschoolcrisisinterventions.
WorkshopSatisfaction
1057
4.45
.71
1053
4.52
.70
1056
4.51
.73
1056
4.24
.80
1056
4.48
.71
1056
4.52
.69
8.Thecontentwasclearandunderstandable.
1054
4.62
.70
9.Workshopmaterialswerewellorganized.
1056
4.67
.70
10.Thetrainer(s)was/werewellorganized.
1056
4.71
.68
11.Workshopmaterialsfacilitatedparticipation
amongparticipants.
12.Thetrainer(s)facilitatedparticipationamong
participants.
13.Thisworkshopincreasedmyknowledge.
1020
4.63
.59
1020
4.71
.59
978
4.72
.56
14.Iwillbeabletoapplytheinformation/skills
learnedtomyprofessionalduties.
15.Irecommendthisworkshop.
16.Irecommendthis/thesetrainer(s).
978
4.63
.59
974
4.69
.62
890
4.75
.63
*Allitemsona1‐5scale,with1meaningstronglydisagreeand5meaningstronglyagree.
WorkshopEffectonParticipants’AttitudesTowardSchoolCrisisWork
Table7offersdescriptivestatisticsforthepre‐andpost‐workshopquestionsaskedof
participantstoassesstheirattitudestowardcrisispreventionandpreparedness.The
overallmeanattitudetowardcrisisinterventionandpreventionworkincreased
significantly(becamemorefavorable,t(492)=21.28,p<.000,etasquared=.48)fromthe
pre‐test(M=3.09,SD=.78)tothepost‐test(M=3.73;SD=.58).Notably,significant
increasesinattitudeswereseenacrossallthreeitems.
Anexplorationoftheassociationofdemographicfactorswithchangesinattitudefounda
significantrelationshipbetweentheamountoftimespentinthecurrentprofessionand
gainsinattitude(F(3,487)=5.16,p=.000,etasquared=.03),withthosereportingfewer
yearsintheprofessionmakingsignificantlylargergainsinattitudetowardcrisis
preventionandinterventionthanthosewithmoreyears(0yearsM=.85,SD=.65;1‐5years
M=.73,SD=.66;6‐10yearsM=.57,SD=.69;11ormoreM=.52,SD=.65).Further,therewere
significantdifferencesfoundforgainsinattitudes(F(3,487)=3.68,p=.012,etasquared=
.03)accordingtopreviousexperiencewithschoolcrisistraining;Thosewith1‐5prior
traininghours(M=.88,SD=.79)significantlymorelikelytoexperiencegainsinattitudes
thanparticipantswith11ormorehoursoftraining(M=.57,SD=.61).Therewerealso
significantdifferencesfoundbasedonreportedoccupation(F(4,483)=2.75,p=.042).
Althoughposthoctestsrevealednoindividualgrouplevelsignificantdifferences,health
professionalsdidnotexperienceanychangeinattitudes(M=.00,SD=.73)whileallother
occupationsexperiencedmorepositiveattitudechange.Therewerenosignificanteffects
foundindifferenceofattitudechangebasedontheparticipant’sstudentstatus
(t(447)=4.84,p=.118).
Table7.Workshop#2Participants’AttitudesTowardCrisisInterventionona1–5
Scale,WithHigherScoresIndicatingMorePositiveAttitudes
N
Pretest
Posttest
Question
Mean SD Mean SD
1.Howanxiouswouldyoufeelifyouwere
493 3.23 .92 3.86 .64
requiredtoconductaschoolcrisisintervention?
2.Howconfidentareyouinyourabilitytoknow
493 3.36 .90 3.92 .60
whattodoifyouwererequiredtorespondaspart
ofaschoolcrisisresponseteam?
3.Howfearfulareyouthatyoumightmakea
mistakeduringaschoolcrisisintervention?
493
2.69
.91
3.42
.82
WorkshopEffectonParticipantSchoolCrisisWorkKnowledge
Workshop2participantresponsesindicatedlarge,significantincreasesinknowledge
acrossallparticipants(t(478)=16.28,p=.000,etasquared=.36)fromthepre‐test(M=6.48
outof13;SD=1.71)tothepost‐test(M=7.93outof13;SD=1.48).Therewereno
significantdifferencesfoundbasedonparticipantdemographicdatafortherelationship
betweentheamountofknowledgegainedantheamountoftimespentinthecurrent
profession(F(3,473)=.939,p=.422),occupation(F(3,470)=1.59,p=.191),studentstatus
(t(434)=1.646,p=.101),andtheamountofpreviousschoolcrisistraining(F(3,473)=1.401,
p=.242).
SummaryofFindings
Insummary,bothworkshop1and2wereassociatedwithlarge,significantincreasesin
attitudesrelatedtoparticipants’professionalrolesincrisispreventionandintervention
Further,bothworkshop1and2participantsreportedahighlevelofsatisfactionwith
workshopcontent,achievementofobjectives,trainerpreparedness,materials,knowledge
gained,andapplicationoftheinformation.