ProgramEvaluationSummary
September2012throughApril2013
ToevaluateworkshopeffectivenessofthePREPaRE2ndeditioncurriculum,collectionof
participantsatisfactionandpre‐andpost‐workshopdataisastandardelementofall
workshopofferings.Thisdocumentsummarizestheprogramevaluationdatacollected
fromSeptember2012throughApril2013.
Workshop1
Ofthepossible855responses,therewere537(63%)participantswithvalidpre‐andpost‐
testdatadefinedaspre‐andpost‐testscompletedwith>50%ofitemscomplete,andno
usererrors(i.e.,redundantresponses,answeringtoomanyitems).Eightyparticipants
(9.4%)weremissingpretestdata,132(15.4%)weremissingposttestdata,94(11%)were
missingbothpreandposttestdata,and12(1.4%)hadinvalidpreandpost‐testdatabased
on>50%ofitemsunansweredorusererrorsincompletion.
DemographicInformation
Table1offersdescriptivestatisticsofparticipantdemographicinformation.The
distributionofyearsintheparticipants’currentprofessionwas:0years(10.6%;n=91),1‐5
years(28.2%;n=241),6‐10years(17.5%;n=150),11ormoreyears(23.5%,n=201),and
172responses(20.1%)weremissing.
ThenumberofpriorschoolcrisistraininghoursWorkshop1participantsreportedvaried,
withthemajorityofparticipantsindicatingtheyhadmorethan10hours(32.4%,n=277)or
0hours(27.6%,n=236);oftheremainingparticipants,7.8%(n=67)had1‐5hoursofprior
schoolcrisistraining,11.6%(n=99)had6‐10hours,and176responses(20.6%)were
missing.
Almosthalfoftherespondentsreportedhavingnocommunityoragency(non‐school)
crisis‐relatedtraining(45%,n=385),however15.2%(n=130)reportedhavingmorethan
10hours.Theremainingrespondentsvariedinreportednon‐schoolcrisis‐relatedtraining
as6.4%(n=55)had1‐5hoursofpriortraining,10.8%(n=92)had6‐10hours,and22.6%
(n=193)weremissing.
Table1.DemographicDataofParticipantsfortheCrisisPreventionand
PreparednessWorkshop(Workshop#1,N=855).
Workshop#1
Demographics
N
Percent
Occupation
MentalHealthProfessionals
374
43.7
SchoolPsychologist
202
23.6
SchoolSocialWorker
37
4.3
AgencySocialWorker
3
0.4
SchoolCounselor
115
13.5
OtherSchoolBasedMentalHealth
13
1.5
OtherCommunityBasedMentalHealth
4
0.5
Educators
252
29.5
GeneralEducationTeacher
24
2.8
SpecialEducationTeacher
12
1.4
SchoolAdministrator
151
17.7
DistrictAdministrator
53
6.2
UniversityProfessor
12
1.4
Health‐Care
11
1.3
SafetyOfficer
8
0.9
Other
31
3.6
Missing
179
20.9
GraduateStudent
119
13.9
Intern(SchoolPsychology)
33
3.9
Race/Ethnicity
Asian
9
1.1
BlackorAfricanAmerican
65
7.6
White
558
65.3
Hispanic/Latino
15
1.8
Other(AmericanIndian,PacificIslander,Other)
14
1.7
Missing
194
22.7
WorkshopSatisfaction
Oftheavailable565Workshop1Evaluationforms,561includedcompleteinformation.
Overall,participantsreportedahighdegreeofsatisfactionwiththeirworkshopexperience
(M=4.54,SD=.55;Questions1,12‐20)andworkshopobjectives(Questions2‐11;M=4.35,
SD=.56).SpecificquestionsandparticipantresponsesaresummarizedinTable2.
Table2.ParticipantWorkshopSatisfactionRatingsfortheCrisisPreventionand
PreparednessWorkshop(Workshop#1).
Workshop#1
Question
N
Mean
SD
CrisisTeam
1.Theobjectiveswereclearlystated.
562
4.64
.63
TheworkshopobjectiveswereclearlymetandIcannowidentifythe:
2.Fourcharacteristicsofacrisisevent.
563
4.33
.71
3.KeyconceptsassociatedwiththePREPaREacronym.
562
4.42
.71
4.Fourphasesofcrisismanagement.
562
4.31
.79
5.Threeconceptsrelatedtocrimepreventionthrough
environmentaldesign.
559
4.44
.68
6.Fourelementsrelatedtopromotingpsychological
safety.
560
4.28
.75
7.Purposeofacomprehensivesafetyteam.
561
4.52
.62
8.MajorfunctionsoftheIncidentCommandSystem
(ICS).
559
4.32
.72
9.Guidingprinciplesincrisisplandevelopment.
560
4.32
.66
10.Differencebetweenthecrisisteamresponseplan
andtheschoolstaffresponseplan.
561
4.24
.77
11.Threestrategiesforexaminingeffectivenessof
crisispreventionandpreparedness.
556
4.31
.69
Workshopsatisfaction
12.Thecontentwasclearandunderstandable.
523
4.47
.71
13.Workshopmaterialswerewellorganized.
521
4.55
.72
14.Thetrainer(s)was/werewellorganized.
520
4.58
.71
15.Workshopmaterialsfacilitatedparticipationamong
participants.
508
4.39
.79
16.Thetrainer(s)facilitatedparticipationamong
participants.
507
4.47
.72
17.Thisworkshopincreasedmyknowledge.
505
4.59
.62
18.Iwillbeabletoapplytheinformation/skillslearned
tomyprofessionalduties.
505
4.48
.63
19.Irecommendthisworkshop.
500
4.48
.72
20.Irecommendthis/thesetrainer(s).
492
4.57
.69
WorkshopEffectonParticipants’AttitudesTowardSchoolCrisisWork
Table3offersdescriptivestatisticsforthepre‐andpost‐workshopquestionsaskedof
participantstoassesstheirattitudestowardcrisispreventionandpreparedness.Despite
participantsstartingoffwithapositiveoverallmeanattitudetowardcrisispreventionand
preparednesswork,scoresstillincreasedsignificantly,t(540)=17.50,p<.001,etasquared
=.36fromthepre‐test(M=3.42outof5;SD=.53)tothepost‐test(M=3.83;SD=.59).When
examiningindividualitems,participantsreportedsignificantchanges(p<.001)onthreeout
ofthefourquestions.Theonlyquestionthatdidnothaveasignificantchangeinvolved
perceptionsoftheimportanceofschoolcrisispreventionandpreparednessknowledgeand
skills,whichwasquitehightobeginwith(M=4.70,SD=0.56).Thelargestgaininattitude
wasseenforitem1,whichindicatesparticipantsfeltmoreknowledgeableaboutschool
crisispreventionandpreparednessafterparticipatinginthisworkshop,t(547)=25.29,
p<.001,etasquared=.54.
Anexplorationoftheassociationofdemographicfactorswithchangesinattitudefound
moderate,significantdifferencesbetweenparticipantsinattitudetowardcrisisprevention
andpreparednessasafunctionofyearsspentintheircurrentprofession,F(3,528)=8.98,
p<.001,etasquared=0.05),withthosewhoreportednoexperience(Mdiff=.64,SD=.54)
makingsignificantlylargergainsinattitudethanparticipantswith6‐10yearsexperience
(Mdiff=.32,SD=.56)andthosewith11ormoreyearsexperience(Mdiff=.29,SD=.51).In
addition,participantswith1‐5yearsexperience(Mdiff=.47,SD=.56)hadsignificantlylarger
gainsinattitudethanthosewith11ormoreyearsexperience.Thiswasfurtherexplained
bythemoderatebutsignificantdifferencefoundbetweenstudentsandworking
professionals,t(507)=3.421,p<.01,etasquared=.02,whichindicatedstudentsreported
morepositivechangesinattitudetowardcrisisprevention(Mdiff=.58,SD=.58)than
professionals(Mdiff=.38,SD=.55).Thenumberofschoolcrisistraininghourswas
moderatelyrelatedtochangesinattitude,F(3,528)=12.64,p<.01,etasquared=.07.
Respondentsreportingnopreviousschoolcrisistrainingindicatedsignificantlyhigher
changesinattitude(Mdiff=.60,SD=.56)thanthosewith1‐5hoursofschoolcrisistraining
(Mdiff=.38,SD=.60)andthosewith11ormorehours(Mdiff=.27,SD=.52).Participantswith6‐
10hoursofpriorschoolcrisistrainingdidnotdiffersignificantlyfromothers(Mdiff=.42,
SD=.50).Nosignificantdifferencewasfoundbasedonnumberofhoursofnon‐school
crisis‐relatedtraining.Thereweremoderate,significantdifferencesbetweenparticipants
reportingdifferentoccupationsF(4,522)=4.11,p<.01,etasquared=.03).Mentalhealth
professionals(M=.49,SD=.50)reportedmorepositivechangesinattitudethaneducators
(M=.28,SD=.63).
Table3.Workshop#1Participants'AttitudesTowardPreventionandPreparedness
Pretest
Posttest
Question
N
Mean
SD Mean
SD
1.Howknowledgeableareyouaboutschool 548
2.45 0.74 3.36
0.75
crisispreventionandpreparedness
2.Howconfidentareyouinyourabilityto
548
2.97 0.98 3.47
0.82
collaboratewithotherstodevelopa
comprehensiveschoolcrisisresponse
managementplan?
3.Howenthusiasticareyoutocollaborate
547
3.55 0.89 3.82
0.88
withotherstodevelopacomprehensive
schoolcrisisresponsemanagementplan?
4.Howimportanttodoyoufeelschoolcrisis 541
4.70 0.56 4.67
0.66
preventionandpreparednessknowledge
andskillsareintoday'sschools?
*Allitemsona5‐pointscale,withhigherscoresindicatingmorepositiveattitudes(e.g.,
1=Notatallknowledgeableto5=Extremelyknowledgeable).
WorkshopEffectonParticipantSchoolCrisisWorkKnowledge
Workshop1participantresponsesindicatedlarge,significantincreasesinknowledge,
t(490)=36.62,p<.001,etasquared=.73frompre‐test(M=5.34outof10,SD=1.48)topost‐
test(M=8.44,SD=1.56).Therewasasmall,butsignificantdifferencefoundbetween
participantsinknowledgegainedincrisispreventionandpreparednessasafunctionof
yearsspentintheircurrentprofession,F(3,624)=2.61,p=.05,etasquared=.01.Individuals
withnoexperienceintheircurrentoccupation(M=2.46,SD=3.22)hadsignificantlyhigher
increasesinknowledgethanthosewith11ormoreyearsofexperience(M=1.21,SD=3.90).
Therewasasmall,butsignificantdifferencefoundbetweenparticipantsasafunctionof
overallexperienceworkinginaschoolsetting,F(625,3)=5.01,p<.01,etasquared=.02,such
thatindividualswithnoexperienceinaschoolsetting(M=2.65,SD=2.89)and1‐5yearsof
experienceinaschoolsetting(M=2.27,SD=3.21)hadgreatergainsinknowledgethan
thosewith11ormoreyearsofexperienceinaschoolsetting(M=1.12,SD=3.78).Therewas
nosignificantdifferenceinknowledgegainedasafunctionofpriorschoolcrisis‐related
training,F(3,622)=1.22,p=ns)orpriorcommunityoragencycrisis‐relatedtraining,F(3,
604)=1.28,p=ns.Therewasasmall,butsignificantdifferenceinknowledgegainedasa
functionofoccupation,F(4,615)=2.729,p<.05,etasquared=.02withmentalhealth
professionals(M=1.95,SD=3.30)exhibitingsignificantlyhighergainsthaneducators
(M=1.05,SD=3.85).Nootheroccupationalgroupcomparisonsdifferedsignificantly.
WORKSHOP2
Ofthepossible857responses,575(67%)participantshadvalidpre‐andposttestsdefined
aspre‐andpost‐testswith>50%ofitemscompleted,andnousererrors(i.e.,redundant
responses,answeringtoomanyitems).Eight‐sixparticipants(10%)weremissingpretest
data,110(12.8%)weremissingposttestdata,57(6.7%)weremissingbothpreandpost
testdataand29(3.4%)hadinvalidpreandpost‐testdata,basedon>50%ofitems
unansweredorusererrorsincompletion.
DemographicInformation
Table4offersdescriptivestatisticsofparticipantdemographicinformationforall
Workshop2participants.Forthisworkshop,thedistributionofyearsofexperienceinthe
currentprofessionwas:0years(9.3%,n=80),1‐5years(26.5%,n=227),6‐10years
(19.3%,n=165),11ormoreyears(27.8%,n=238),and147responses(17.2%)were
missing.Overallexperienceinschoolswas:0years(4.4%,n=38),1‐5years(19.3%,
n=165),6‐10years(16.9%,n=145),11ormoreyears(42.2%,n=362),and147responses
(17.2%)weremissing.Themajorityofparticipants(31.3%,n=268)had11ormorehours
ofpriorschoolcrisisinterventiontrainingpriortotheworkshop;oftheremaining
participants,23.9%(n=205)hadnopriortraining,7.8%(n=67)had1‐5hours,18.9%
(n=162)had6‐10hours,and18.1%(155)oftheresponsesweremissing.Themajorityof
participants(46.7%,n=400)hadnopriornon‐schoolcrisis‐relatedtraining;ofthe
remainingparticipants,5.7%(n=49)had1‐5hours,9%(n=77)had6‐10hours,19.4%
(n=166)had11ormorehours,and19.3%(n=165)oftheresponsesweremissing.
Table4.DemographicDataofParticipantsfortheCrisisInterventionandRecovery
Workshop(Workshop#2,N=857).
Workshop#2
Demographics
N
Percent
Occupation
MentalHealthProfessionals
583
68.0
SchoolPsychologist
245
28.6
SchoolSocialWorker
82
9.6
AgencySocialWorker
4
0.5
SchoolCounselor
199
23.2
OtherSchoolBasedMentalHealth
42
4.9
OtherCommunityBasedMentalHealth
11
1.3
Educators
82
9.6
GeneralEducationTeacher
10
1.2
SpecialEducationTeacher
6
.7
SchoolAdministrator
33
3.9
DistrictAdministrator
29
3.4
UniversityProfessor
4
.5
Health‐Care
22
2.6
SafetyOfficer
0
0
Other
20
2.3
Missing
179
20.9
GraduateStudent
118
13.8
Intern(SchoolPsychology)
45
5.3
Race/Ethnicity
Asian
14
1.6
BlackorAfricanAmerican
56
6.5
White
508
59.3
Hispanic/Latino
67
7.8
Other(AmericanIndian,PacificIslander,Other)
14
1.8
Missing
197
23.0
WorkshopSatisfaction
Ofthe653Workshop2Evaluationforms,649includedresponsesto>50%ofthe
satisfactionquestions.Overall,totalparticipantsatisfactionforWorkshop2washigh
(M=4.53,SD=.53).Participantsappearedtoreportequallyhighsatisfactionforworkshop
experience(M=4.59,SD=.58)andworkshopobjectives(M=4.45,SD=.56).Specific
questionsandparticipantresponsesaresummarizedinTable5.
Table5.ParticipantWorkshopSatisfactionRatingsfortheCrisisInterventionand
RecoveryWorkshop.
Workshop#2
Question
N
Mean
SD
WorkshopObjectives
1.Theworkshopobjectiveswereclearlystated.
649
4.64
0.74
TheworkshopobjectivesweremetandIamnowabletodothefollowing:
2.Reportimprovedattitudestoward,and
readinesstoprovide,schoolcrisisintervention.
649
4.4
3.Identifyvariablesthatdeterminethenumberof
individualslikelytobetraumatizedbyagiven
crisis.
646
4.52
4.Identifytheschoolcrisisinterventionsspecified
bythePREPaREacronym.
647
4.52
5.Identifyhowschoolcrisisinterventionfitsinto
themultidisciplinary(NIMS/ICS)schoolcrisis
response.
646
4.31
6.Statethetriagevariablesthatpredict
psychologicaltrauma.
648
4.44
7.Matchthedegreeofpsychologicaltraumarisk
totheappropriateschoolcrisisinterventions.
648
4.51
WorkshopSatisfaction
8.Thecontentwasclearandunderstandable.
649
4.58
9.Workshopmaterialswerewellorganized.
646
4.66
10.Thetrainer(s)was/werewellorganized.
649
4.69
11.Workshopmaterialsfacilitatedparticipation
amongparticipants.
647
4.51
12.Thetrainer(s)facilitatedparticipationamong
participants.
646
4.6
13.Thisworkshopincreasedmyknowledge.
620
4.61
14.Iwillbeabletoapplytheinformation/skills
learnedtomyprofessionalduties.
614
4.56
15.Irecommendthisworkshop.
610
4.56
16.Irecommendthis/thesetrainers.
565
4.63
0.75
0.65
0.66
0.74
0.71
0.61
0.69
0.66
0.62
0.78
0.71
0.71
0.65
0.77
0.72
*Allitemsona1‐5scale,with1meaningstronglydisagreeand5meaningstronglyagree.
WorkshopEffectonParticipants’AttitudesTowardSchoolCrisisWork
Table6offersdescriptivestatisticsforthepre‐andpost‐workshopquestionsaskedof
participantstoassesstheirattitudestowardcrisispreventionandpreparedness.The
overallmeanattitudetowardcrisisinterventionandpreventionworkincreased
significantlyt(600)=24.74,p<.01,etasquared=0.34suchthatattitudeimprovedfrom
pretest(M=3.07,SD=.76)toposttest(M=3.74,SD=.59).Significantincreasesinattitude
wereseenacrossallthreeitems.
Anexplorationoftheassociationofdemographicfactorswithchangesinattitudefounda
smallbutsignificantrelationshipbetweentheamountoftimespentinthecurrent
professionandgainsinattitude,F(3,593)=5.23,p<.001,etasquared=.03.Respondents
whohad<5yearsofexperienceintheircurrentprofessionhadsignificantlylargergains
thanthosewith11ormoreyearsexperience(0yearMdiff=.85,SD=.65;1‐5yearsMdiff=.74,
SD=.67;6‐10yearsMdiff=.63,SD=.55;11ormoreyearsMdiff=.54,SD=.69).Priorschool
crisistrainingwasalsoassociatedwithsmallbutsignificantdifferencesfoundforgainsin
attitude,F(3,590)=7.93,p<.001,etasquared=.04.Respondentswithnopriorschoolrelated
crisistrainingandthosewith6‐10hoursoftrainingreportedsignificantlygreatergains
thanthosewith11ormorehoursoftraining(0hoursMdiff=.77,SD=.72;1‐5hoursMdiff
=.62,SD=.57;6‐10hoursMdiff=.81,SD=.57;11ormorehoursMdiff=.51,SD=.65).Therewas
asmallbutsignificantdifferenceinchangesinattitudebasedonpriornon‐schoolcrisis‐
relatedtrainingF(3,584)=8.82,p<.001,etasquared=0.04,aswell.Thosewhoreportedno
priornon‐schoolcrisis‐relatedtrainingdemonstratedhighergainsthanthosewith11or
morehoursofexperience(0hoursMdiff=.76,SD=.68;1‐5hoursMdiff=.71,SD=.70;6‐10
hoursMdiff=.66,SD=.53;11ormorehoursMdiff=.42,SD=.58).Therewerenosignificant
differencesbasedonreportedoccupationF(3,593)=.90,p>ns.Therewasalsoasmallbut
significantdifferenceinchangeinattitudebasedonstudentstatusF(1,547)=12.52,
p=<.001,etasquared=0.02.Graduatestudentshadgreaterattitudegains(Mdiff=.87,
SD=.68)thanindividualswhowerenotcurrentlyingraduateschool(Mdiff=.62,SD=.65).
Internstatuswasalsoassociatedwithsmallbutsignificantgainsinattitude,F(1,
242)=19.00,p<.001,etasquared=0.07,suchthatthosewhowereinterningatthetimeof
theworkshop(Mdiff=1.05,SD=.64)hadgreatergainsthanthosewhowerenot(Mdiff=.59,
SD=.59).
Table6.Participants’AttitudesTowardCrisisIntervention(Workshop#2).
Pretest
Posttest
Question
Mean
SD
Mean
SD
1.Howanxiouswouldyoufeelifyou
wererequiredtoconductaschool
crisisintervention?(1=Extremely
anxiousto5=Notatallanxious)
3.23
0.91
3.82
0.71
2.Howconfidentareyouinyour
abilitytoknowwhattodoifyou
wererequiredtorespondaspartofa
schoolcrisisresponseteam?(1=Not
atallconfidentto5=Extremely
3.36
0.91
3.93
0.61
confident)
3.Howfearfulareyouthatyoumight
makeamistakeduringaschoolcrisis
intervention?(1=Extremelyfearful
to5=Notatallfearful)
2.69
0.88
3.48
0.84
*Allitemsona1‐5scale,withhigherscoresindicatingmorepositiveattitudes(e.g.,
1=Extremelyanxiousto5=Notatallanxious).
WorkshopEffectonParticipantSchoolCrisisWorkKnowledge
Workshop2participantresponsesindicatedsignificantincreasesinknowledge,
t(539)=17.32,p<.001,etasquared=.36fromthepretest(M=6.46outof13,SD=1.61)tothe
posttest(M=7.76,SD=1.52).Therewerenosignificantdifferencesfoundbasedon
participantdemographicdatafortherelationshipbetweentheamountofknowledge
gainedandanyofthefollowing:currentoccupation,F(3,529)=.686,p=ns,studentstatus,
F(1,491)=.246,p=ns,internshipstatus,F(1,213)=.066,p=ns),theamountoftimespentin
currentprofession,F(3,533)=.79,p=ns,overallexperienceinschools,F(3,533)=.153,
p=ns),priorschoolrelatedcrisistraining,F(3,529)=.115,p=ns,andpriornonschoolcrisis‐
relatedtraining,F(3,522)=.190,p=ns).
SummaryofFindings
Insummary,bothWorkshop1and2resultedinlarge,significantincreasesinattitudesand
knowledgerelatedtocrisispreventionandintervention.Respondentsalsoreportedahigh
degreeofsatisfactionwiththeirworkshopexperienceandworkshopobjectivesincluding
contentsatisfaction,knowledgegained,applicability,andlikelihoodofrecommendingthe
workshopandtrainerstoothers.