ProgramEvaluationSummary
February2014throughAugust2014
nd
InordertoevaluatethePREPaRE2 editioncurriculumworkshopeffectiveness,
participantsatisfactionandpre‐andpost‐workshopdataarecollectedasastandard
elementofallworkshopsoffered.Thisdocumentsummarizestheprogramevaluationdata
collectedfromFebruary2014throughAugust2014.
Workshop1
Ofthepossible1229responses,therewere811(66.0%)participantswithvalidpre‐and
post‐testdatadefinedaspre‐andpost‐testscompletedwith>50%ofitemscomplete,and
nousererrors(i.e.,redundantresponses,answeringtoomanyitems).
Onehundredandeightparticipants(8.8%)weremissingpretestdata,217(17.7%)were
missingposttestdata,87(7.1%)weremissingbothpreandposttestdata,and5(0.40%)
hadinvalidpreandpost‐testdatabasedon>50%ofitemsunansweredorusererrorsin
completion.
DemographicInformation
Table1offersdescriptivestatisticsofparticipantdemographicinformation.The
distributionofyearsintheparticipants’currentprofessionwas:0years(12.7%;n=156),1‐
5years(24.5%;n=301),6‐10years(17.0%;n=209),11ormoreyears(29.7%,n=365),and
198responses(16.1%)weremissing.
Thenumberofpriorschoolcrisistraininghoursreportedvaried,withthemajorityof
participantsindicatingtheyhad11ormorehours(27.5%,n=338)or0hours(31.1%,
n=382).Oftheremainingparticipants,9.9%(n=122)had1‐5hoursofpriorschoolcrisis
training,14.8%(n=182)had6‐10hours,and205responses(16.7%)weremissing.
Almosthalfoftherespondentsreportedhavingnocommunityoragency(non‐school)
crisis‐relatedtraining(43.7%,n=537),however20.6%(n=253)reportedhaving11or
morehours.Theremainingrespondentsvariedinreportednon‐schoolcrisis‐related
trainingas7.3%(n=90)had1‐5hoursofpriortraining,10.9%(n=134)had6‐10hours,
and17.5%(n=215)weremissing.
Table1.DemographicDataofParticipantsfortheCrisisPreventionand
PreparednessWorkshop(Workshop#1,N=1229).
Workshop#1
Demographics
N
Percent
Occupation
MentalHealthProfessional
630
51.3
SchoolPsychologist
333
27.1
SchoolSocialWorker
89
7.2
AgencySocialWorker
6
0.5
SchoolCounselor
157
12.8
OtherSchoolBasedMentalHealth
22
1.8
OtherCommunityBasedMentalHealth
23
1.9
SchoolFaculty/Administration
249
20.3
GeneralEducationTeacher
28
2.3
SpecialEducationTeacher
16
1.3
SchoolAdministrator
135
11.0
DistrictAdministrator
47
3.8
UniversityProfessor
23
1.9
Health‐CareProfessional
26
2.1
Security/LawEnforcement
38
3.1
Other
75
6.1
Missing
211
17.2
GraduateStudent
225
18.3
Intern(SchoolPsychology)
38
3.1
Race/Ethnicity
Asian
13
1.1
BlackorAfricanAmerican
101
8.2
White
733
59.6
Hispanic/Latino
101
8.2
NativeHawaiianorOtherPacificIslander
4
.3
AmericanIndian/AlaskaNative
6
.5
Other
10
.8
Missing
261
21.2
WorkshopSatisfaction
Overall,participantsreportedahighdegreeofsatisfactionwiththeirworkshopexperience
(M=4.56,SD=.61;Questions1,12‐20)andworkshopobjectives(Questions2‐11;M=4.36,
SD=.53).SpecificquestionsandparticipantresponsesaresummarizedinTable2.
Table2.ParticipantWorkshopSatisfactionRatingsfortheCrisisPreventionand
PreparednessWorkshop(Workshop#1).
Workshop#1
Question
N
Mean
SD
CrisisTeam
1.Theobjectiveswereclearlystated.
822
4.64
.66
TheworkshopobjectiveswereclearlymetandIcannowidentifythe:
2.Fourcharacteristicsofacrisisevent.
820
3.KeyconceptsassociatedwiththePREPaREacronym.
821
4.Fourphasesofcrisismanagement.
818
5.Threeconceptsrelatedtocrimepreventionthrough
environmentaldesign.
819
6.Fourelementsrelatedtopromotingpsychological
safety.
821
7.Purposeofacomprehensivesafetyteam.
819
8.MajorfunctionsoftheIncidentCommandSystem
(ICS).
819
9.Guidingprinciplesincrisisplandevelopment.
818
10.Differencebetweenthecrisisteamresponseplan
andtheschoolstaffresponseplan.
819
11.Threestrategiesforexaminingeffectivenessof
crisispreventionandpreparedness.
803
Workshopsatisfaction
12.Thecontentwasclearandunderstandable.
767
13.Workshopmaterialswerewellorganized.
767
14.Thetrainer(s)was/werewellorganized.
765
15.Workshopmaterialsfacilitatedparticipationamong
participants.
763
16.Thetrainer(s)facilitatedparticipationamong
participants.
761
17.Thisworkshopincreasedmyknowledge.
760
18.Iwillbeabletoapplytheinformation/skillslearned
tomyprofessionalduties.
759
19.Irecommendthisworkshop.
757
20.Irecommendthis/thesetrainer(s).
747
4.37
4.44
4.34
.68
.67
.71
4.47
.67
4.26
4.46
.74
.66
4.32
4.28
.74
.73
4.23
.78
4.31
.70
4.53
4.64
4.62
.72
.65
.68
4.43
.84
4.52
4.58
.76
.71
4.44
4.50
4.59
.77
.79
.77
WorkshopEffectonParticipants’AttitudesTowardSchoolCrisisWork
Table3offersdescriptivestatisticsforthepre‐andpost‐workshopquestionsaskedof
participantstoassesstheirattitudestowardcrisispreventionandpreparedness.Despite
participantsstartingoffwithapositiveoverallmeanattitudetowardcrisispreventionand
preparednesswork,scoresstillincreasedsignificantly,t(793)=26.82,p<.001,etasquared
=.47fromthepre‐test(M=3.40outof5;SD=.57)tothepost‐test(M=3.92;SD=.57).When
examiningindividualitems,participantsreportedsignificantchanges(p<.001)inallfour
questions.Thelargestgaininattitudewasseenforitem1,whichindicatesparticipantsfelt
moreknowledgeableaboutschoolcrisispreventionandpreparednessafterparticipatingin
thisworkshop,t(796)=32.36,p<.001,etasquared=.56.Anexplorationoftheassociationof
demographicfactorswithchangesinattitudefoundmoderate,significantdifferences
betweenparticipantsinattitudetowardcrisispreventionandpreparednessasafunction
ofyearsspentintheircurrentprofession,F(3,782)=15.98,p<.001,etasquared=0.06,with
thosewhoreportednoexperience(Mdiff=.77,SD=.54)makingsignificantlylargergainsin
attitudethanparticipantswith1to5yearsofexperience(Mdiff=.52,SD=.48),6‐10years
experience(Mdiff=.46,SD=.54)andthosewith11ormoreyearsexperience(Mdiff=.39,
SD=.55).Thiswasfurtherexplainedbythemoderate,butsignificantdifferencefound
betweenstudentsandworkingprofessionals,F(1,740)=41.21,p<.001,etasquared=0.05,
whichindicatedstudentsreportedmorepositivechangesinattitudetowardcrisis
prevention(Mdiff=.73,SD=.51)thanprofessionals(Mdiff=.44,SD=.53).Therewasasmall,
butsignificantrelationshipbetweenthenumberofschoolcrisistraininghoursandchanges
inattitude,F(3,779)=8.69,p<.001,etasquared=.03.Respondentsreportingnoprevious
schoolcrisistrainingindicatedsignificantlyhigherchangesinattitude(Mdiff=.61,SD=.60)
thanthosewith1‐5hoursofschoolcrisistraining(Mdiff=.54,SD=.52)andthosewith11or
morehours(Mdiff=.38,SD=.48).Participantswith6‐10hoursofpriorschoolcrisistraining
didnotdiffersignificantlyfromothers(Mdiff=.51,SD=.47).Asmall,butsignificant
differencewasalsofoundforindividualsreportingnopreviousnon‐schoolcrisis‐related
trainingandthosereporting6‐10hoursofpriortraining,F(3,774)=3.40,p<.05,eta
squared=.01.Therewerenosignificantdifferencesbetweenparticipantsreporting
differentoccupations.
Table3.Workshop#1Participants'AttitudesTowardPreventionandPreparedness
Pretest
Posttest
Question
N
Mean SD Mean
SD
1.Howknowledgeableareyouaboutschool 797 2.43 .818 3.45
.745
crisispreventionandpreparedness?
2.Howconfidentareyouinyourabilityto
795
2.91 1.03 3.61
.820
collaboratewithotherstodevelopa
comprehensiveschoolcrisisresponse
managementplan?
3.Howenthusiasticareyoutocollaborate
796 3.58 .907 3.92
.876
withotherstodevelopacomprehensive
schoolcrisisresponsemanagementplan?
4.Howimportanttodoyoufeelschoolcrisis 796 4.69 .611 4.71
.588
preventionandpreparednessknowledge
andskillsareintoday'sschools?
*Allitemsona5‐pointscale,withhigherscoresindicatingmorepositiveattitudes(e.g.,
1=Notatallknowledgeableto5=Extremelyknowledgeable).
WorkshopEffectonParticipantSchoolCrisisWorkKnowledge
Workshop1participantresponsesindicatedlarge,significantincreasesinknowledge,
t(734)=44.03,p<.001,etasquared=0.72frompre‐test(M=5.42outof10,SD=1.59)topost‐
test(M=8.29,SD=1.64).Therewasnosignificantdifferencefoundbetweenknowledge
gainedandparticipants’yearsofexperienceintheircurrentoccupation,F(3,809)=1.42,
p=ns.Therewasasignificantdifferenceinknowledgegainedfoundbetweenparticipants
asafunctionoftheiroverallexperienceworkinginaschoolsetting,F(3,724)=2.74,,p<.05,
etasquared=.011.Therewasasignificantdifferenceinknowledgegainedasafunctionof
priorschoolcrisis‐relatedtraining,F(3,720)=7.16,p<.001,etasquared=.03.Respondents
reportingnopreviousschoolcrisistrainingindicatedsignificantlyhigherchangesin
knowledge(Mdiff=3.18,SD=1.73)thanthosewith11ormorehours(Mdiff=2.48,SD=1.71).A
significantimpactwasalsofoundbetweenindividualsbasedontheirpriorcommunityor
agencycrisis‐relatedtrainingF(3,717)=2.63p<.05,etasquared=.01,withparticipantswho
hadnoprioragencycrisisrelatedtraining(Mdiff=3.00,SD=1.77)gainingsignificantlymore
thanindividualswhohad11ormorehoursoftraining(Mdiff=2.58,SD=1.77).Therewasno
significantdifferenceinknowledgegainedasafunctionofoccupation,F(4,713)=2.17,
p=ns.
WORKSHOP2
Ofthepossible1299responses,942(72.5%)participantshadvalidpre‐andposttests
definedaspre‐andpost‐testswith>50%ofitemscompleted,andnousererrors(i.e.,
redundantresponses,answeringtoomanyitems).Onehundredthirty‐twoparticipants
(10.2%)weremissingpretestdata,141(10.9%)weremissingposttestdata,83(6.4%)
weremissingbothpreandposttestdataand1(.1%)hadinvalidpreandpost‐testdata,
basedon>50%ofitemsunansweredorusererrorsincompletion.
DemographicInformation
Table4offersdescriptivestatisticsofparticipantdemographicinformationforall
Workshop2participants.Forthisworkshop,thedistributionofyearsofexperienceinthe
currentprofessionwas:0years(12.5%,n=163),1‐5years(23.3%,n=303),6‐10years
(16.3%,n=212),11ormoreyears(29.9%,n=389),and232responses(17.9%)were
missing.Overallexperienceinschoolswas:0years(6.1%,n=79),1‐5years(21.0%,
n=273),6‐10years(16.0%,n=208),11ormoreyears(38.7%,n=503),and236responses
(18.2%)weremissing.Themajorityofparticipants(31.2%,n=405)had11ormorehours
ofpriorschoolcrisisinterventiontrainingpriortotheworkshop;oftheremaining
participants,19.6%(n=255)hadnopriortraining,6.5%(n=85)had1‐5hours,23.8%
(n=309)had6‐10hours,and18.9%(245)oftheresponsesweremissing.Themajorityof
participants(45.3%,n=588)hadnopriornon‐schoolcrisis‐relatedtraining;ofthe
remainingparticipants,6.2%(n=81)had1‐5hours,11.4%(n=148)had6‐10hours,18.2%
(n=236)had11ormorehours,and18.9%(n=246)oftheresponsesweremissing.
Table4.DemographicDataofParticipantsfortheCrisisInterventionandRecovery
Workshop(Workshop#2,N=1299).
Workshop#2
Demographics
N
Percent
Occupation
MentalHealthProfessionals
853
65.7
SchoolPsychologist
521
40.1
SchoolSocialWorker
70
5.4
AgencySocialWorker
5
0.4
SchoolCounselor
224
17.2
OtherSchoolBasedMentalHealth
26
2.0
OtherCommunityBasedMentalHealth
7
0.5
Educators
129
9.9
GeneralEducationTeacher
28
2.2
SpecialEducationTeacher
18
1.4
SchoolAdministrator
36
2.8
DistrictAdministrator
26
2.0
UniversityProfessor
21
1.6
Health‐Care
27
2.1
SafetyOfficer
12
0.9
Other
36
2.8
Missing
242
18.6
GraduateStudent
251
19.3
Intern(SchoolPsychology)
75
5.8
Race/Ethnicity
Asian
25
1.9
BlackorAfricanAmerican
68
5.2
White
774
59.6
Hispanic/Latino
102
7.9
Other(AmericanIndian,PacificIslander,Other)
26
2.0
Missing
304
23.4
WorkshopSatisfaction
Overall,totalparticipantsatisfactionforWorkshop2washigh(M=4.64,SD=.44).
Participantsappearedtoreportequallyhighsatisfactionforworkshopobjectives(M=4.53,
SD=.49)andworkshopsatisfaction(M=4.70,SD=.48).Specificquestionsandparticipant
responsesaresummarizedinTable5.
Table5.ParticipantWorkshopSatisfactionRatingsfortheCrisisInterventionand
RecoveryWorkshop.
Workshop#2
Question
N
Mean
SD
WorkshopObjectives
1.Theworkshopobjectiveswereclearlystated.
921
4.76
0.59
TheworkshopobjectivesweremetandIamnowabletodothefollowing:
2.Reportimprovedattitudestoward,and
readinesstoprovide,schoolcrisisintervention.
921
4.52
3.Identifyvariablesthatdeterminethenumberof
individualslikelytobetraumatizedbyagiven
crisis.
921
4.60
4.Identifytheschoolcrisisinterventionsspecified
921
4.55
bythePREPaREacronym.
5.Identifyhowschoolcrisisinterventionfitsinto
themultidisciplinary(NIMS/ICS)schoolcrisis
response.
920
4.35
6.Statethetriagevariablesthatpredict
psychologicaltrauma.
919
4.56
7.Matchthedegreeofpsychologicaltraumarisk
totheappropriateschoolcrisisinterventions.
920
4.57
WorkshopSatisfaction
8.Thecontentwasclearandunderstandable.
920
4.67
9.Workshopmaterialswerewellorganized.
919
4.71
10.Thetrainer(s)was/werewellorganized.
920
4.74
11.Workshopmaterialsfacilitatedparticipation
amongparticipants.
919
4.63
12.Thetrainer(s)facilitatedparticipationamong
participants.
918
4.70
13.Thisworkshopincreasedmyknowledge.
890
4.73
14.Iwillbeabletoapplytheinformation/skills
learnedtomyprofessionalduties.
891
4.61
15.Irecommendthisworkshop.
883
4.69
16.Irecommendthis/thesetrainers.
819
4.73
0.61
0.57
0.61
0.67
0.60
0.58
0.60
0.60
0.55
0.63
0.59
0.59
0.59
0.63
0.63
*Allitemsona1‐5scale,with1meaningstronglydisagreeand5meaningstronglyagree.
WorkshopEffectonParticipants’AttitudesTowardSchoolCrisisWork
Table6offersdescriptivestatisticsforthepre‐andpost‐workshopquestionsaskedof
participantstoassesstheirattitudestowardcrisispreventionandpreparedness.The
overallmeanattitudetowardcrisisinterventionandpreventionworkincreased
significantlyt(935)=34.08,p<.001,etasquared=.55,suchthatattitudeimprovedfrom
pretest(M=2.99,SD=.81)toposttest(M=3.75,SD=.58).Significantincreasesinattitude
wereseenacrossallthreeitems.Anexplorationoftheassociationofdemographicfactors
withchangesinattitudefoundalargesignificantrelationshipbetweentheamountoftime
spentinthecurrentprofessionandgainsinattitude,F(3,917)=30.96,p<.001,etasquared
=.09.Respondentswhohadnoexperienceintheircurrentprofession(Mdiff=1.18,SD=.64)
hadsignificantlylargergainsthanthosewith1‐5years(Mdiff=.78,SD=.61),6‐10years(Mdiff
=.66,SD=.67),and11ormoreyears(Mdiff=.58,SD=.68).Priorschoolcrisistrainingwasalso
associatedwithsignificantdifferencesfoundforgainsinattitude,F(3,909)=23.03,p<.001,
etasquared=0.07.Respondentswith11ormorehoursofpriorschoolrelatedcrisis
reportedsignificantlyfewergains(Mdiff=.54,SD=.64)thanthosewithnopriorschoolcrisis
training(Mdiff=.96,SD=.70),andforparticipantswith1‐5hours(Mdiff=.95,SD=.70)and6‐
10hoursoftraining(Mdiff=.82,SD=.63).Therewasalsoasignificantdifferenceinchanges
inattitudebasedonpriornon‐schoolcrisis‐relatedtrainingF(3,907)=15.36,p<.001,eta
squared=0.05.Thosewhoreportednopriornon‐schoolcrisis‐relatedtraining(Mdiff=.86,
SD=.68)andthosewhoreported1to5hoursoftraining(Mdiff=.94,SD=.67)demonstrated
highergainsthanthosewith6ormorehoursofexperience(6‐10hoursMdiff=.59,SD=.74;
11ormorehoursMdiff=.54,SD=.57).Therewasnosignificantdifferenceinchangesin
attitudesbasedonreportedoccupationF(4,908)=1.86,p=ns.Therewasalsoasignificant
differenceinchangeinattitudebasedonstudentstatusF(1,850)=67.79,p=<.001,eta
squared=.07.Graduatestudentshadgreaterattitudegains(Mdiff=1.08,SD=.67)than
individualswhowerenotcurrentlyingraduateschool(Mdiff=.66,SD=.64).
Table6.Participants’AttitudesTowardCrisisIntervention(Workshop#2).
Pretest
Posttest
Question
Mean
SD
Mean
SD
1.Howanxiouswouldyoufeelifyou
wererequiredtoconductaschool
crisisintervention?(1=Extremely
anxiousto5=Notatallanxious)
3.12
0.95
3.84
0.64
2.Howconfidentareyouinyour
abilitytoknowwhattodoifyou
wererequiredtorespondaspartofa
schoolcrisisresponseteam?(1=Not
atallconfidentto5=Extremely
confident)
3.23
0.92
3.89
0.61
3.Howfearfulareyouthatyoumight
makeamistakeduringaschoolcrisis
intervention?(1=Extremelyfearful
to5=Notatallfearful)
3.62
0.94
3.51
0.82
*Allitemsona1‐5scale,withhigherscoresindicatingmorepositiveattitudes(e.g.,
1=Extremelyanxiousto5=Notatallanxious).
WorkshopEffectonParticipantSchoolCrisisWorkKnowledge
Workshop2participantresponsesindicatedsignificantincreasesinknowledge,
t(839)=34.49,p<.001,etasquared=.59fromthepretest(M=7.82outof13,SD=2.08)tothe
posttest(M=10.73,SD=1.94).Therewerenosignificantdifferencesfoundbasedon
participantdemographicdatafortherelationshipbetweentheamountofknowledge
gainedandanyofthefollowing:studentstatus,F(1,760)=.384,p=ns,internshipstatus,F(1,
451)=.341,p=ns,theamountoftimespentincurrentprofession,F(3,822)=.426p=ns,
overallexperience,F(3,819)=.339,p=ns,priorschoolrelatedcrisistraining,F(3,
815)=.816,p=ns,andpriornon‐schoolcrisis‐relatedtraining,F(3,812)=.838,p=ns.There
wasasmall,butsignificantdifferenceintermsoftheparticipants’currentoccupation,F(4,
815)=2.50p=.05,etasquared=.012,withschoolfaculty/administration(Mdiff=3.33,
SD=3.44)makingthemostgainsandhealthcareprofessionalsmakingtheleastgains(Mdiff
=1.55,SD=2.01).
SummaryofFindings
Insummary,bothWorkshop1and2resultedinsignificantincreasesinattitudesand
knowledgerelatedtocrisispreventionandintervention.Respondentsalsoreportedahigh
degreeofsatisfactionwiththeirworkshopexperienceandworkshopobjectivesincluding
contentsatisfaction,knowledgegained,applicability,andlikelihoodofrecommendingthe
workshopandtrainerstoothers.