ProgramEvaluationSummary
February2013throughDecember2013
nd
InordertoevaluatethePREPaRE2 editioncurriculumworkshopeffectiveness,
participantsatisfactionandpre‐andpost‐workshopdataiscollectedasastandard
elementofallworkshopsoffered.Thisdocumentsummarizestheprogramevaluationdata
collectedfromFebruary2013throughDecember2013.
Workshop1
Ofthepossible1357responses,therewere876(64.6%)participantswithvalidpre‐and
post‐testdatadefinedaspre‐andpost‐testscompletedwith>50%ofitemscomplete,and
nousererrors(i.e.,redundantresponses,answeringtoomanyitems).Ninetyparticipants
(6.6%)weremissingpretestdata,177(13.0%)weremissingposttestdata,200(14.7%)
weremissingbothpreandposttestdata,and14(1.6%)hadinvalidpreandpost‐testdata
basedon>50%ofitemsunansweredorusererrorsincompletion.
DemographicInformation
Table1offersdescriptivestatisticsofparticipantdemographicinformation.The
distributionofyearsintheparticipants’currentprofessionwas:0years(11.7%;n=159),1‐
5years(23.2%;n=315),6‐10years(14.0%;n=190),11ormoreyears(29.1%,n=395),and
298responses(22.0%)weremissing.
ThenumberofpriorschoolcrisistraininghoursWorkshop1participantsreportedvaried,
withthemajorityofparticipantsindicatingtheyhadmorethan10hours(23.1%,n=314)or
0hours(28.7%,n=389);oftheremainingparticipants,10.6%(n=144)had1‐5hoursof
priorschoolcrisistraining,15.6%(n=212)had6‐10hours,and298responses(22%)were
missing.
Almosthalfoftherespondentsreportedhavingnocommunityoragency(non‐school)
crisis‐relatedtraining(47.5%,n=645);however,15.3%(n=208)reportedhavingmore
than10hours.Theremainingrespondentsvariedinreportednon‐schoolcrisis‐related
trainingas6.0%(n=82)had1‐5hoursofpriortraining,8.9%(n=121)had6‐10hours,and
22.2%(n=301)weremissing.
Table1.DemographicDataofParticipantsfortheCrisisPreventionand
PreparednessWorkshop(Workshop#1,N=855).
Workshop#1
Demographics
N
Percent
Occupation
MentalHealthProfessionals
738
54.4
SchoolPsychologist
526
38.8
SchoolSocialWorker
68
5.0
AgencySocialWorker
1
0.1
SchoolCounselor
123
9.1
OtherSchoolBasedMentalHealth
12
.9
OtherCommunityBasedMentalHealth
8
.6
Educators
233
17.2
GeneralEducationTeacher
20
1.5
SpecialEducationTeacher
27
2.0
SchoolAdministrator
116
8.5
DistrictAdministrator
47
3.5
UniversityProfessor
23
1.7
Health‐Care
32
2.4
SafetyOfficer
15
1.4
Other
33
2.4
Missing
306
22.5
GraduateStudent
242
17.8
Intern(SchoolPsychology)
61
4.5
Race/Ethnicity
Asian
22
1.6
BlackorAfricanAmerican
47
3.5
White
894
65.9
Hispanic/Latino
54
4.0
Other(AmericanIndian,PacificIslander,Other)
10
.7
Missing
326
24.0
WorkshopSatisfaction
Oftheavailable913Workshop1Evaluationforms,827includedcompleteinformation.
Overall,participantsreportedahighdegreeofsatisfactionwiththeirworkshopexperience
(M=4.57,SD=.49;Questions1,12‐20)andworkshopobjectives(Questions2‐11;M=4.34,
SD=.53).SpecificquestionsandparticipantresponsesaresummarizedinTable2.
2
Table2.ParticipantWorkshopSatisfactionRatingsfortheCrisisPreventionand
PreparednessWorkshop(Workshop#1).
Workshop#1
Question
N
Mean
SD
CrisisTeam
1.Theobjectiveswereclearlystated.
913
4.67
.60
TheworkshopobjectiveswereclearlymetandIcannowidentifythe:
2.Fourcharacteristicsofacrisisevent.
911
4.35
.67
3.KeyconceptsassociatedwiththePREPaREacronym.
910
4.43
.65
4.Fourphasesofcrisismanagement.
910
4.31
.71
5.Threeconceptsrelatedtocrimepreventionthrough
environmentaldesign.
910
4.49
.64
6.Fourelementsrelatedtopromotingpsychological
safety.
910
4.28
.74
7.Purposeofacomprehensivesafetyteam.
910
4.52
.61
8.MajorfunctionsoftheIncidentCommandSystem
(ICS).
909
4.30
.71
9.Guidingprinciplesincrisisplandevelopment.
908
4.25
.67
10.Differencebetweenthecrisisteamresponseplan
andtheschoolstaffresponseplan.
906
4.23
.77
11.Threestrategiesforexaminingeffectivenessof
crisispreventionandpreparedness.
905
4.25
.72
Workshopsatisfaction
12.Thecontentwasclearandunderstandable.
871
4.56
.61
13.Workshopmaterialswerewellorganized.
868
4.66
.59
14.Thetrainer(s)was/werewellorganized.
869
4.65
.60
15.Workshopmaterialsfacilitatedparticipationamong
participants.
863
4.45
.72
16.Thetrainer(s)facilitatedparticipationamong
participants.
857
4.51
.69
17.Thisworkshopincreasedmyknowledge.
856
4.57
.63
18.Iwillbeabletoapplytheinformation/skillslearned
tomyprofessionalduties.
856
4.44
.68
19.Irecommendthisworkshop.
855
4.51
.70
20.Irecommendthis/thesetrainer(s).
843
4.56
.69
*Allitemsona1‐5scale,with1meaningstronglydisagreeand5meaningstronglyagree.
WorkshopEffectonParticipants’AttitudesTowardSchoolCrisisWork
Table3offersdescriptivestatisticsforthepre‐andpost‐workshopquestionsaskedof
participantstoassesstheirattitudestowardcrisispreventionandpreparedness.Despite
participantsstartingoffwithapositiveoverallmeanattitudetowardcrisispreventionand
preparednesswork,scoresstillincreasedsignificantly,t(866)=28.40,p<.001,etasquared
=.48fromthepre‐test(M=3.37outof5;SD=.55)tothepost‐test(M=3.87;SD=.55).When
3
examiningindividualitems,participantsreportedsignificantchanges(p<.001)onallfour
questions.Thelargestgaininattitudewasseenforitem1,whichindicatesparticipantsfelt
moreknowledgeableaboutschoolcrisispreventionandpreparednessafterparticipatingin
thisworkshop,t(879)=36.03,p<.001,etasquared=.60.
Anexplorationoftheassociationofdemographicfactorswithchangesinattitudefound
moderate,significantdifferencesbetweenparticipantsinattitudetowardcrisisprevention
andpreparednessasafunctionofyearsspentintheircurrentprofession,F(3,
856)=20.769,p<.001,etasquared=0.07),withthosewhoreportednoexperience(Mdiff=.80,
SD=.51)makingsignificantlylargergainsinattitudethanparticipantswith1to5yearsof
experience(Mdiff=.52,SD=.55),6‐10yearsexperience(Mdiff=.41,SD=.47)andthosewith11
ormoreyearsexperience(Mdiff=.42,SD=.47).Thiswasfurtherexplainedbythemoderate,
butsignificantdifferencefoundbetweenstudentsandworkingprofessionals,F(1,
791)=77.17,p<.001,etasquared=0.09),whichindicatedstudentsreportedmorepositive
changesinattitudetowardcrisisprevention(Mdiff=.79,SD=.53)thanprofessionals(Mdiff
=.43,SD=.49).Thenumberofschoolcrisistraininghourswasmoderatelyrelatedto
changesinattitude,F(3,528)=12.64,p<.01,etasquared=.07.Respondentsreportingno
previousschoolcrisistrainingindicatedsignificantlyhigherchangesinattitude(Mdiff=.60,
SD=.56)thanthosewith1‐5hoursofschoolcrisistraining(Mdiff=.38,SD=.60)andthose
with11ormorehours(Mdiff=.27,SD=.52).Participantswith6‐10hoursofpriorschool
crisistrainingdidnotdiffersignificantlyfromothers(Mdiff=.42,SD=.50).Nosignificant
differencewasfoundbasedonnumberofhoursofnon‐schoolcrisis‐relatedtraining.
Thereweremoderate,significantdifferencesbetweenparticipantsreportingdifferent
occupationsF(4,522)=4.11,p<.01,etasquared=.03).Mentalhealthprofessionals(M=.49,
SD=.50)reportedmorepositivechangesinattitudethaneducators(M=.28,SD=.63).
Table3.Workshop#1Participants'AttitudesTowardPreventionandPreparedness
Pretest
Posttest
Question
N
Mean
SD Mean
SD
1.Howknowledgeableareyouaboutschool 880
2.34 0.78 3.34
0.71
crisispreventionandpreparedness
2.Howconfidentareyouinyourabilityto
880
2.90 1.03 3.56
0.80
collaboratewithotherstodevelopa
comprehensiveschoolcrisisresponse
managementplan?
3.Howenthusiasticareyoutocollaborate
874
3.57 0.88 3.86
0.86
withotherstodevelopacomprehensive
schoolcrisisresponsemanagementplan?
4.Howimportanttodoyoufeelschoolcrisis 872
4.66 0.63 4.71
0.56
preventionandpreparednessknowledge
andskillsareintoday'sschools?
*Allitemsona5‐pointscale,withhigherscoresindicatingmorepositiveattitudes(e.g.,
1=Notatallknowledgeableto5=Extremelyknowledgeable).
4
WorkshopEffectonParticipantSchoolCrisisWorkKnowledge
Workshop1participantresponsesindicatedlarge,significantincreasesinknowledge,
t(821)=50.50,p<.001,etasquared=0.76frompre‐test(M=5.48outof10,SD=1.46)topost‐
test(M=8.40,SD=1.45).Therewasasmall,butsignificantdifferencefoundbetween
participantsinknowledgegainedincrisispreventionandpreparednessasafunctionof
yearsspentintheircurrentprofession,F(3,811)=3.37,p=.02,etasquared=.01.Individuals
with6to10yearsofexperience(M=3.28,SD=1.70)hadsignificantlyhigherincreasesin
knowledgethanthosewith1to5yearsexperienceintheircurrentoccupation(M=2.76,
SD=1.60).Therewasnosignificantdifferencefoundbetweenparticipantsasafunctionof
overallexperienceworkinginaschoolsettingF(3,809)=1.42,p=ns.Therewasno
significantdifferenceinknowledgegainedasafunctionofpriorschoolcrisis‐related
training,F(3,811)=1.60,p=nsorpriorcommunityoragencycrisis‐relatedtraining,F(3,
810)=.32,p=ns.Therewasalsonosignificantdifferenceinknowledgegainedasafunction
ofoccupation,F(4,804)=.78,p=ns.
WORKSHOP2
Ofthepossible1456responses,982(67.4%)participantshadvalidpre‐andposttests
definedaspre‐andpost‐testswith>50%ofitemscompleted,andnousererrors(i.e.,
redundantresponses,answeringtoomanyitems).Onehundredfortyparticipants(9.6%)
weremissingpretestdata,174(12%)weremissingposttestdata,132(9.1%)weremissing
bothpreandposttestdata,and28(1.9%)hadinvalidpreandpost‐testdata,basedon
>50%ofitemsunansweredorusererrorsincompletion.
DemographicInformation
Table4offersdescriptivestatisticsofparticipantdemographicinformationforall
Workshop2participants.Forthisworkshop,thedistributionofyearsofexperienceinthe
currentprofessionwas:0years(10.4%,n=151),1‐5years(24.7%,n=359),6‐10years
(16.3%,n=238),11ormoreyears(29.2%,n=425),and283responses(19.4%)were
missing.Overallexperienceinschoolswas:0years(4.3%,n=63),1‐5years(21%,n=306),
6‐10years(14.9%,n=217),11ormoreyears(40.2%,n=585);285responses(19.6%)were
missing.Themajorityofparticipants(35.7%,n=520)had11ormorehoursofpriorschool
crisisinterventiontrainingpriortotheworkshop;oftheremainingparticipants,18.6%
(n=271)hadnopriortraining,6.3%(n=91)had1‐5hours,19.5%(n=284)had6‐10hours,
and19.9%(290)oftheresponsesweremissing.Themajorityofparticipants(45.5%,
n=663)hadnopriornon‐schoolcrisis‐relatedtraining;oftheremainingparticipants,5.9%
(n=86)had1‐5hours,10.2%(n=149)had6‐10hours,18.1%(n=263)had11ormore
hours,and20.3%(n=295)oftheresponsesweremissing.
5
Table4.DemographicDataofParticipantsfortheCrisisInterventionandRecovery
Workshop(Workshop#2,N=857).
Workshop#2
Demographics
N
Percent
Occupation
MentalHealthProfessionals
980
67.3
SchoolPsychologist
459
31.5
SchoolSocialWorker
123
8.4
AgencySocialWorker
10
0.7
SchoolCounselor
344
23.6
OtherSchoolBasedMentalHealth
28
1.9
OtherCommunityBasedMentalHealth
16
1.1
Educators
108
7.4
GeneralEducationTeacher
6
0.4
SpecialEducationTeacher
9
0.6
SchoolAdministrator
43
3.0
DistrictAdministrator
35
2.4
UniversityProfessor
15
1.0
Health‐Care
12
.8
SafetyOfficer
47
3.2
Other
19
1.3
Missing
290
19.9
GraduateStudent
213
14.6
Intern(SchoolPsychology)
94
6.5
Race/Ethnicity
Asian
26
1.8
BlackorAfricanAmerican
133
9.1
White
882
60.6
Hispanic/Latino
135
9.3
Other(AmericanIndian,PacificIslander,Other)
21
1.5
Missing
259
17.8
WorkshopSatisfaction
Ofthe1095Workshop2Evaluationforms,927includedcompleteinformation.Overall,
totalparticipantsatisfactionforWorkshop2washigh(M=4.57,SD=.48).Participants
appearedtoreportequallyhighsatisfactionforworkshopexperience(M=4.64,SD=.50)and
workshopobjectives(M=4.43,SD=.56).Specificquestionsandparticipantresponsesare
summarizedinTable5.
6
Table5.ParticipantWorkshopSatisfactionRatingsfortheCrisisInterventionand
RecoveryWorkshop.
Workshop#2
Question
N
Mean
SD
WorkshopObjectives
1.Theworkshopobjectiveswereclearlystated.
1095
4.67
.74
TheworkshopobjectivesweremetandIamnowabletodothefollowing:
2.Reportimprovedattitudestoward,and
readinesstoprovide,schoolcrisisintervention.
1093
4.41
3.Identifyvariablesthatdeterminethenumberof
individualslikelytobetraumatizedbyagiven
crisis.
1094
4.52
4.Identifytheschoolcrisisinterventionsspecified
1094
4.47
bythePREPaREacronym.
5.Identifyhowschoolcrisisinterventionfitsinto
themultidisciplinary(NIMS/ICS)schoolcrisis
response.
1090
4.28
6.Statethetriagevariablesthatpredict
psychologicaltrauma.
1093
4.44
7.Matchthedegreeofpsychologicaltraumarisk
totheappropriateschoolcrisisinterventions.
1093
4.51
WorkshopSatisfaction
8.Thecontentwasclearandunderstandable.
1093
4.61
9.Workshopmaterialswerewellorganized.
1092
4.67
10.Thetrainer(s)was/werewellorganized.
1093
4.68
11.Workshopmaterialsfacilitatedparticipation
amongparticipants.
1094
4.57
12.Thetrainer(s)facilitatedparticipationamong
participants.
1092
4.62
13.Thisworkshopincreasedmyknowledge.
1029
4.63
14.Iwillbeabletoapplytheinformation/skills
learnedtomyprofessionalduties.
1028
4.57
15.Irecommendthisworkshop.
1026
4.65
16.Irecommendthis/thesetrainers.
941
4.71
0.73
0.69
0.71
0.71
0.72
0.66
0.64
0.65
0.64
0.71
0.65
0.67
0.62
0.66
0.61
*Allitemsona1‐5scale,with1meaningstronglydisagreeand5meaningstronglyagree.
WorkshopEffectonParticipants’AttitudesTowardSchoolCrisisWork
Table6offersdescriptivestatisticsforthepre‐andpost‐workshopquestionsaskedof
participantstoassesstheirattitudestowardcrisispreventionandpreparedness.The
overallmeanattitudetowardcrisisinterventionandpreventionworkincreased
significantlyt(1004)=29.35,p<.001,etasquared=0.46suchthatattitudeimprovedfrom
7
pretest(M=3.05,SD=.80)toposttest(M=3.74,SD=.63).Significantincreasesinattitude
wereseenacrossallthreeitems.
Anexplorationoftheassociationofdemographicfactorswithchangesinattitudefounda
small,butsignificantrelationshipbetweentheamountoftimespentinthecurrent
professionandgainsinattitude,F(3,989)=11.54,p<.001,etasquared=.03.Respondents
whohadnoexperienceintheircurrentprofessionhadsignificantlylargergainsthanthose
with1ormoreyearsexperience(0yearMdiff=.96,SD=.76;1‐5yearsMdiff=.74,SD=.76;6‐10
yearsMdiff=.64,SD=.73;11ormoreyearsMdiff=.56,SD=.68).Priorschoolcrisistraining
wasalsoassociatedwithsignificantdifferencesfoundforgainsinattitude,F(3,982)=13.15,
p<.001,etasquared=.04.Respondentswithnopriorschoolrelatedcrisisreported
significantlygreatergainsthanthosewith1ormorehoursoftrainingandthosewith6to
10hoursoftrainingdifferedsignificantlyfromthosewithmorethan10hours(0hours
Mdiff=.93,SD=.79;1‐5hoursMdiff=.66,SD=.76;6‐10hoursMdiff=.74,SD=.69;11ormore
hoursMdiff=.56,SD=.70).Therewasasmall,butsignificantdifferenceinchangesin
attitudebasedonpriornon‐schoolcrisis‐relatedtrainingF(3,984)=10.33,p<.001,eta
squared=0.03,aswell.Thosewhoreportednopriornon‐schoolcrisis‐relatedtraining
demonstratedhighergainsthanthosewith6ormorehoursofexperience(0hoursMdiff
=.79,SD=.75;1‐5hoursMdiff=.73,SD=.67;6‐10hoursMdiff=.60,SD=.67;11ormorehours
Mdiff=.48,SD=.72).Therewasasignificantdifferenceinchangesinattitudesbasedon
reportedoccupationF(4,981)=3.93,p=.004.Mentalhealthprofessionalshadsignificantly
highergainsthanschoolfaculty/administration(mentalhealthprofessionalsMdiff=.71,
SD=.71,schoolfaculty/administrationMdiff=.48,SD=.80,healthcareprofessionalsMdiff=.90,
SD=.80,security/lawenforcementMdiff=.44,SD=.96,otherMdiff=.98,SD=.96).Therewasalsoa
significantdifferenceinchangeinattitudebasedonstudentstatusF(1,904)=46.34,
p=<.001,etasquared=0.05.Graduatestudentshadgreaterattitudegains(Mdiff=1.02,
SD=.77)thanindividualswhowerenotcurrentlyingraduateschool(Mdiff=.63,SD=.70).
Internstatuswasalsoassociatedwithsmall,butsignificantgainsinattitude,F(1,
534)=13.31,p<.001,etasquared=0.02,suchthatthosewhowereinterningatthetimeof
theworkshophadgreatergains(Mdiff=1.02,SD=.73)thanthosewhowerenot(Mdiff=.70,
SD=.74).
Table6.Participants’AttitudesTowardCrisisIntervention(Workshop#2).
Pretest
Posttest
Mean
SD
Mean
SD
Question
1.Howanxiouswouldyoufeelifyou
wererequiredtoconductaschool
crisisintervention?(1=Extremely
anxiousto5=Notatallanxious)
3.21
0.94
3.81
0.72
2.Howconfidentareyouinyour
abilitytoknowwhattodoifyou
wererequiredtorespondaspartofa
schoolcrisisresponseteam?(1=Not
atallconfidentto5=Extremely
confident)
3.33
0.95
3.88
0.69
8
3.Howfearfulareyouthatyoumight
makeamistakeduringaschoolcrisis
intervention?(1=Extremelyfearful
to5=Notatallfearful)
2.74
0.93
3.52
0.84
*Allitemsona1‐5scale,withhigherscoresindicatingmorepositiveattitudes(e.g.,
1=Extremelyanxiousto5=Notatallanxious).
WorkshopEffectonParticipantSchoolCrisisWorkKnowledge
Workshop2participantresponsesindicatedsignificantincreasesinknowledge,
t(891)=35.35,p<.001,etasquared=.58fromthepretest(M=7.68outof13,SD=2.10)tothe
posttest(M=10.56,SD=2.00).Therewerenosignificantdifferencesfoundbasedon
participantdemographicdatafortherelationshipbetweentheamountofknowledge
gainedandanyofthefollowing:currentoccupation,F(3,870)=2.37,p=ns,studentstatus,
F(1,491)=2.17,p=ns,internshipstatus,F(1,493)=2.53,p=ns),theamountoftimespentin
currentprofession,F(3,876)=1.47,p=ns,overallexperienceinschools,F(3,875)=.431,
p=ns),priorschoolrelatedcrisistraining,F(3,871)=1.59,p=ns,andpriornon‐schoolcrisis‐
relatedtraining,F(3,873)=.37,p=ns).
SummaryofFindings
Insummary,bothWorkshop1and2resultedinlarge,significantincreasesinattitudesand
knowledgerelatedtocrisispreventionandintervention.Respondentsalsoreportedahigh
degreeofsatisfactionwiththeirworkshopexperienceandworkshopobjectivesincluding
contentsatisfaction,knowledgegained,applicability,andlikelihoodofrecommendingthe
workshopandtrainerstoothers.
9