C O R P O R AT I O N
The Development
and Pilot Testing of the
ENGAGED Toolkit
Joie D. Acosta, Anita Chandra, Vivian L. Towe, Yandong Zhao, Yangxu Lu
This toolkit was sponsored by the Ford Foundation.
For more information on this publication, visit www.rand.org/t/TL202
Published by the RAND Corporation, Santa Monica, Calif.
© Copyright 2016 RAND Corporation
R® is a registered trademark.
Limited Print and Electronic Distribution Rights
This document and trademark(s) contained herein are protected by law. This representation of RAND
intellectual property is provided for noncommercial use only. Unauthorized posting of this publication
online is prohibited. Permission is given to duplicate this document for personal use only, as long as it
is unaltered and complete. Permission is required from RAND to reproduce, or reuse in another form, any of
its research documents for commercial use. For information on reprint and linking permissions, please visit
www.rand.org/pubs/permissions.html.
The RAND Corporation is a research organization that develops solutions to public policy challenges to help make
communities throughout the world safer and more secure, healthier and more prosperous. RAND is nonprofit,
nonpartisan, and committed to the public interest.
RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors.
Support RAND
Make a tax-deductible charitable contribution at
www.rand.org/giving/contribute
www.rand.org
Contents
ToolkitPurpose............................................................................................................................................1
MethodsUsedtoDeveloptheToolkit.........................................................................................................1
LiteratureReview....................................................................................................................................2
KeyInformantInterviewsandFocusGroups..........................................................................................3
OperationalModelforInvolvingNGOsinDisasterPreparedness,Response,andRecovery.................5
U.S.PilotTestoftheToolkit........................................................................................................................7
NextSteps..................................................................................................................................................10
References.................................................................................................................................................10
iii
ToolkitPurpose
Recentstrategies,suchastheUnitedStates’NationalHealthSecurityStrategyandNationalDisaster
RecoveryFramework,aswellaspolicydocumentsgeneratedbytheInternationalRedCrossandthe
SPHEREProject,haveemphasizedtheimportanceofengagingorganizationsoutsidethegovernmenton
disasterresponseandrecovery.Still,thereisnoofficialpolicy,documentation,orguidanceforhow
nongovernmentalorganizations(NGOs)shouldleadorworkwithgovernmentthroughthedisaster
responseandrecoveryphases.1Further,thereislimitedinternationalguidanceontheroleofNGOsin
disasterresponseandrecovery.WhileNGOsprovidecriticalsocial,economic,andhealthservices,there
isevidencetosuggestthattheireffectivenesscouldbeenhancediftherelevantorganizationswere
moreformallyengagedinrecoveryeffortsandbetterintegratedintoplanningatthelocalandstate
levels.
Since2013,theRANDCorporationhaspartneredwiththeChineseAcademyofScienceandTechnology
forDevelopment(CASTED)toexploretheroleofNGOs(specifically,voluntaryassociations;
philanthropicorganizations,suchasfoundationsanddonororganizations;advocacygroups;community
groups;andbusinesses)inChinaandtheUnitedStatesindisasterresponseandrecovery.TheFord
FoundationfundedRANDandCASTEDtoidentifythekeyassetsandskillsofNGOsandtodevelopa
toolkit(theEnablingNonGovernmentalAgenciestoGetEngagedinDisasters,orENGAGED,Toolkit)to
facilitatemore-reliableandmore-effectiveNGOinvolvementindisasterpreparedness,response,and
recovery.
ThetoolkitassistspublichealthandemergencyplannersandNGOstakeholders(includinginternational
aidorganizations,in-countryorganizations,philanthropicorganizations,andbusinesses)indetermining
thecapacityandcapabilityofparticularNGOsfordisasterresponseandrecovery.
Inaddition,thetoolkitfillsanimportantgapinknowledgeandunderstandingaboutthekeyelements
thatdriveNGOparticipation.AlthoughparticipationfromNGOsiscritical,leveragingtheseactorshas
beenhighlyvariableacrosscommunities,andinmanycases,organizationsthatcomposethissectorface
inadequatepolicyandfinancialsupporttoparticipateeffectively(Cutteretal.,2006;Waugh,2006;
Chandra&Acosta,2009;Acosta,Chandra,Sleeper,&Springgate,2011;Moore,Chandra,&Feeney,
2013).WithincreasingdisastersinboththeUnitedStatesandChina(e.g.,2013U.S.tornados,Hurricane
Sandy,andthe2010and2013earthquakesinsouthwestChina),havingtoolstosupportengagementof
thesevoluntary,social,andphilanthropicNGOsiscriticalbecausetheyareinstrumentalcontributorsto
disasterresponseandrecoveryefforts.
MethodsUsedtoDeveloptheToolkit
TodeveloptheENGAGEDtoolkit,weusedtwomethods:
1) literaturereview,whichincludedsynthesisfrompriorRANDanalyses,aswellasaliterature
search
Forthepurposesofthistoolkit,werefertodisastersasbothnaturaldisasters(suchashurricanes,floods,and
tornados)andcomplexhumanitariandisasters(suchasoilspills,nucleardisasters,anddisplacedpopulations).
1
1
2) keyinformantinterviewsandfocusgroupswithorganizationsintheUnitedStatestocapture
theirrolesandexpectationsforNGOsandtounderstandwhereopportunitiesformorerobust
engagementmayexist.
ThesemethodshelpedidentifythedimensionsofNGOengagementthatcontributetosuccessful
disasterresponseandrecovery,aswellasNGOcapacitiesandcapabilitiesinneedofimprovementfor
optimalengagementindisasterresponseandrecovery.
Wethenusedtheinformationcollectedthroughthesetwomethods(i.e.,thedimensionsofNGO
engagementandareasinwhichNGOsneedtoimprove)todevelopastoryboardanddraftthetoolkit
content.Onceadraftwasdeveloped,wepilottestedthetoolkitinthreecommunitiesintheUnited
States.Inthissection,webrieflydescribehoweachmethodinformedthefinaltoolkit’sdesignand
content.Thesubsequentsectionfocusesonthepilottest.
Notethatthroughoutthedevelopmentofthetoolkit(2012–2016),weengagedwithCASTEDpartnersto
developourmethodsandshareourfindingsfromtheliteraturereviewandinterviewstoreviewthe
findingsfromtheCASTEDteam’sinterviewswithChinesecivilsocietyorganizations(thetermusedto
describeNGOsinChina).Whileourtwoprojectswereconductedindependently,wepartneredand
sharedourworktoultimatelyinformthedevelopmentofatoolkitthatcanbeusedinbothcountries.
RANDandCASTEDworkedtogetherto:
• identifythekeyculturalandsocietalelementsdrivingsocialparticipationindisaster
preparedness,response,andrecoveryintheUnitedStatesandChina,andcreateaconceptual
model(seeAcosta&Chandra,2013)
• usethisconceptualmodeltoguidethedevelopmentofthetoolkit.
Onceadraftofthetoolkitwasdeveloped,RANDthen:
•
•
conductedapilottestofthedrafttoolkittoidentifythekeystepsforstakeholderstousethe
toolinpractice
conductedasurveyofNGOsintheUnitedStatestodeterminewhetherthereisempirical
evidencetosupporttheconceptualmodelusedtoguidethedevelopmentofthetoolkit.
Surveyfindingswillbesharedinafuturepublication.Thisdocumentfocusesonfindingsfromthepilot
testofthetoolkit.AlthoughCASTEDisalsoconductingapilottestofthedrafttoolkitwithcommunities
inChinaandrunningasimilarsurveywithNGOsinChina,thisdocumenthighlightsonlyRAND
contributionsandfocusesmoreexclusivelyonU.S.communities.AsofFebruary2016,CASTEDwasstill
completingitspilottestandsurvey.OnceCASTEDcompletesitswork,thefindingsfromboththeUnited
StatesandChinacanbefurtherintegrated.
LiteratureReview
Toinformthedevelopmentofthetoolkit,wefirstconductedananalysisofpeer-reviewedliterature
(coveringdisastersacrosstheglobe),relevantU.S.policy,andfederalguidancetocharacterizethe
capabilitiesofNGOs,contextualfactorsthatdeterminetheirinvolvementindisasteroperations,andthe
keyservicesNGOsprovideduringdisasterresponseandrecovery. Toidentifypeer-reviewedliterature,
weidentifiedarticlesfrom2000orlaterusingatitlesearchofMedline(PubMed)andPsychInfo
databases.Searchtermsincludednongovernment*ornonprofitorfaith-basedorbusinessor
community-basedANDdisasterorresilienceorpreparednessoremergencyormitigation.Wealso
2
reviewedthefollowingrecentpolicydocumentstoidentifyNGOrolesasoutlinedincurrentpolicyand
federalguidance:theNationalHealthSecurityStrategy,NationalSecurityStrategy,NationalDisaster
RecoveryFramework,HomelandSecurityPresidentialDirective21,GrandChallengesforDisaster
Reduction,All-HazardRiskMitigationPlan,DepartmentofHealthandHumanServices’Pandemic
InfluenzaPlan,andFederalEmergencyManagementAgency’sWholeCommunityEngagementStrategy.
Fromeachdocument,wecatalogedcontent(e.g.,typeofstudy,datacollectionmethod,analytic
approach,andasummaryofkeyfindings),theuniquecapabilitiesofNGOsreferencedineachcitation
(e.g.,flexibility),determinantsofinvolvementindisasteroperations(e.g.,financialconsiderations),
servicesthatNGOsprovidedduringdisasterresponseandrecovery(e.g.,shelter,food),andhowthese
keyservicesdifferedforroutineandemergencytimes.Thecatalogueofdataabstractionformcontents
wassynthesizedintoasinglelistofuniquecapabilities,determinants,andservicesforroutineand
emergencytimes.
Findingsfromtheliteraturereviewwereusedtoinformthedevelopmentofanoperationalmodel
describingNGOinvolvementindisaster(seeFigure1laterinthisdocument)andfocusedonthe
followingfourinterrelatedareas:
• thecurrentpolicylandscapesupportingpartnershipsamongNGOsandbetweenNGOsand
government,aswellasthecommonchallengestodevelopingandleveragingthesepartnerships
• capabilitiesanddeterminantsofNGOcapacitytorespondtodisaster
• servicesofferedbyNGOs,aswellastheirrolesandresponsibilitiesbothduringroutine
operationsandduringdisasterresponseandrecovery
• thepotentiallong-termbenefitsofinvolvingNGOsindisasterresponseandrecovery,which
includebuildingNGOcapacityandencouragingongoingcommunitydevelopment.
Detailsonhowtheliterature,policy,andguidancedocumentswerereviewedandkeyinformation
abstracted,aswellaskeyfindingsfromtheliteraturereview,arecontainedinAcostaandChandra
(2013).
KeyInformantInterviewsandFocusGroups
TolearnmoreaboutNGOs’experienceindisasterresponseandrecovery,theRANDteamconductedsix
90-minutefocusgroups(n=55)or45-minutequalitativeinterviews(n=21;forthoseunabletoattendthe
focusgroup)withNGOandgovernmentalstakeholderscommonlyinvolvedindisasterresponseand
recoveryeffortsintheUnitedStates.Stakeholdersincludedindividualsrepresentinglocalgovernment
(e.g.,citymanagers),faith-basedorganizations(e.g.,Catholiccharities),nonprofits(e.g.,RedCross,
UnitedWay),long-termrecoverycommittees,communityfoundations,businesses(e.g.,Walmart,
chamberofcommercerepresentatives),emergencymanagement(e.g.,FederalEmergency
ManagementAgency),firstresponders(e.g.,fire,police),healthandhealthcare(e.g.,hospitals),public
health(e.g.,localhealthdepartments),andcoalitionsofNGOs(e.g.,localvoluntaryorganizationsactive
indisaster[VOADs]).Weselectedsixsitesthathaverecentlyexperiencedadisaster:Joplin,Missouri
(twofocusgroups,n=13;interviews,n=9);NewYork,NewYork(interviews,n=4);Mobile,Alabama(one
focusgroup,n=11;interview,n=1);Birmingham,Alabama(onefocusgroup,n=13;interviews,n=6);
Biloxi,Mississippi(onefocusgroup,n=9;interview,n=1);andNewOrleans,Louisiana(onefocusgroup,
n=9).Keystakeholdersateachsitewereselectedthroughconsultationwithakeyinformantateachsite.
Intervieweesincludedadiversearrayofcivilsociety(volunteer,community-based,business,and
philanthropicorganizations).
3
TheCASTEDteamconducteditsowninterviewswithNGOsinChina.Tofacilitatecross-countrylessons
learned,wedevelopedtheinterviewandfocusgroupprotocolinpartnershipwithCASTEDsothatwe
couldbothusethesameprotocolwhentalkingwithNGOs.Theprotocolfocusedonthesequestions:
•
HowareNGOsusedindisasters?Whatrolesdotheyplay?Dotheyplanforaroleoristheir
participationadhoc?Whatmotivatestheirparticipation?
•
WhathavebeensuccessfulrolesforNGOs?Whatareashavepresentedchallenges,andwhy?
•
HowhaveNGOsbeenintegratedintoresponseandrecoveryplans?Howisthiscoordinated,
linked,andresourced?
•
HowhastheresponseofNGOsvariedbytypeofdisaster,previouslocaldisasterexperiences,or
otherrelevantfactors?Canwedistillcorerolesorelementsacrossdisasters?
•
WhatdataexistonthebenefitsofinvolvingNGOsindisasterresponseandrecovery?Canwe
identifyproximaloutcomes(e.g.,activeinvolvementofNGOsindisasterplanning)thatmediate
orpredictthesuccessofNGOsindisasterresponseandrecovery?
AnswerswerethendistilledtoidentifyhowNGOsareengagedandinvolvedindisasterresponseand
recoveryefforts,thefacilitatingfactorsandchallengestoinvolvement,theresourcesthatthe
organizationsrelyon,andtheoverallbenefitsoftheirinvolvementinsuchefforts(boththosethatoccur
sequentiallybeforeadisasterresponseandrecoveryand,preferably,thosethatoccuronaregularbasis
toallowforroutinemonitoringoftheseoutcomesandthosethatmediateorpredictthesuccessof
NGOsindisasterresponseandrecovery).
ThefollowingkeythemessurfacedfromtheU.S.interviews:
• SomeofthekeyfactorsthatfacilitateNGOengagementinclude:
o trustamongNGOs,andtrustbetweenNGOsandgovernmentagencies
o havingtheleadershipoftheNGOinvestedindisasterresponseandrecoveryandin
collaboration
o havingincentives(bothfiscalandnonfiscal)tohelpeaseNGOparticipation
o ensuringprepositionedplansforfinancingandcoordinationinadvanceofthedisaster
o havingclearplansthatguideresponsecoordinationandrecoveryoperations
o believinginthereliability("youcancounton")ofNGOstoengageeffectivelyin
responseandrecoveryoperations.
• Duringrecovery,inparticular,factorsthatfacilitatedNGOengagementinclude:
o settingcommunitywiderecoverygoalsthatincludedadefinedroleforNGOs
o usingsocialmediatorecruitvolunteersthatsupporttheworkofNGOsandtheoverall
communityrecovery.
• SomeofthefactorsthatimprovedcoordinationamongNGOsinclude:
o amulti-agencywarehouseforsuppliesthatallowedcoordinationacrossallpointsof
dispensing
o strongcommunityorganizationsactiveindisasterthatwereconnectedtoregionaland
stateVOADs(twoU.S.modelsforengagingNGOsindisaster)
o aCharityTracker,whichallowedfortrackingdonatedgoodsandindividualassistance.
• WealsolearnedthatsomeofthecriticalbarriersthatcontinuetoimpedeeffectiveNGO
participationinclude:
o challengesincommunicationamongNGOsandbetweentheseorganizationsand
governmentagencies(e.g.,intheUnitedStates,thereisnocommonconvenerto
coordinateinformationexchange)
4
o
o
o
o
o
difficultieswithtrustandconflictingperspectivesonparticularNGOreliability
difficultiesbalancingdisasterresponseandrecoveryagainstusualNGOoperations
challengesinfinancingandreimbursement
confusionaboutrolesandresponsibilitiesofNGOsinrecovery
lackofcentralizedinformationaboutclientswhoaccessmultipleservices,leadingto
concernsoverduplication.
KeyissuesthatemergedfromtheinterviewswithChinesecivilsocietyorganizationsweresimilartothe
U.S.findings.Theyincluded:
• lackofinformationaboutwhateachcivilsocietyorganizationdoes,whatassetseachbrings,and
howresourcesareshared
• littleinformationonhowgovernmentshouldworkwithcivilsocietyorganizations
• difficultydeterminingequityinresourceidentificationandsharing
• difficultydeterminingwhenandhowcertaintypesoforganizationsshouldengageacrossthe
disastercycle
• confusionaboutthetypesofneedsineducation,socialwelfare,health,environmental,legal,
andotherdisasterreliefrequirementsandwhatorganizationshoulddo.
Theanalysisofliteratureandkeyinformantinterviewsandfocusgroupsinformedafinaloperational
model(Figure1),whichweoriginallypublishedinAcostaandChandra(2013).Thismodelservedasthe
foundationalframeworkforENGAGEDandisintendedtobeapplicableinboththeUnitedStatesand
China.WealsointegratedsomeofthekeyfindingsfromtheCASTEDteam’sinterviewswithChinesecivil
societyorganizations,whichwebrieflysummarizeinthenextsection.ENGAGEDwasdesignedtohelp
NGOs(orcivilsocietyorganizations)applythismodelbyassessingtheircapacityandcapabilitiesnoted
here.
OperationalModelforInvolvingNGOsinDisasterPreparedness,Response,and
Recovery
ThefollowingisanexcerptfromAcostaandChandra(2013)explainingtheoperationalmodel:
[TheNGOoperationalmodelisbasedonfourinter-relatedkeyareas.]Thefirstareadescribesthe
currentpolicylandscapesupportingpartnershipsbetweenNGOsandgovernmentagenciesand
amongNGOsandthecommonchallengestodevelopingandleveragingthesepartnerships.The
secondareaindicatesthecapabilitiesanddeterminantsofNGOcapacitytorespondtodisaster.The
thirdareadescribestheservicesofferedbyNGOsandtheirrolesandresponsibilities.Thisareain
particularhelpsdefinetheunderlyingpremiseofthemodel—thatNGOsshouldbeinvolvedto
providetheirroutineservicesmorequicklyandtoagreaterandbroaderpopulationduringa
disaster.ThefourthareaunderscorestheimportanceofinvolvingNGOsindisasterresponseand
recoveryforpotentiallong-termbenefits,includingbuildingNGOcapacityandencouragingongoing
communitydevelopment.
TheFigureshowshowthesecomponentsarewovenintothemodel.KeyNGOcapabilitiesinclude
theirknowledgeoflocalneedsandassetsdevelopedovertimethroughtheirrelationshipswith
otherNGOs.Otherdeterminantsoftheircapacityinfluenceinvolvementindeliveringroutineand
disaster-relatedactivities,suchasfinancial(assetlevel,reimbursement)andsocialresponsibility
(mission,motivations).Whenadisasteraffectsacommunity,NGOsapplytheircapabilitiesand
acceleratethedeliveryofservicestoaccommodatenewcommunityneeds.Theyarenotaskedto
conductnewservicesorassumenewresponsibilitiesbuttoincreasetheireffortsbyprovidingmore
5
servicesfasterandtoabroaderpopulation.Duringdisasterrecovery,NGOscontinueproviding
servicesandbegintransitioningfamiliesbackintotheroutineservicedeliverysystem(e.g.,
TemporaryAssistanceforNeedyFamilies).Aspartofcontinuedcommunitydevelopment,these
improvementscanstimulateeconomicrecovery,improveservicedeliveryforfuturedisasters,and
providegeneralstrengtheningofresilience.Lessonslearnedaboutwhatworkedandwhatfailed
duringadisastercanalsoprovidecriticalinformationtofurtherenhanceNGOscapacity.(pp.362–
363)
Figure1.OperationalModelforInvolvingNGOsinDisasterPreparedness,Response,andRecovery
Ongoing&capacity&building&
Stressor(
Other(Determinants(
• Social(responsibility(
• Financial(
considerations(
• Community/(
constituency(needs(
and(values(
• Risk(attitude(and(
industry(norms(
Mo
re&
NGOs(
Ramp(Up(
Services(
Fast
er&
Local&community&
Nongovernmental(
Organizations((
8For(profit(
8Non8profit(
Routine(Services/Planning(
Participate&in/lead&
community&and&disaster&
planning&
(
Provide&services&
• Physical(&(mental(health(
care(
• Short(&(long8term(
housing(
• Employment((
• Case(management(
• Other(social(services(
• Donations(&(volunteer(
management(
• Animal(care(
• Spiritual(&(emotional(care(
Bro
ade
r&Po
p&
Federal,(State,(
and(Local(
Government(
Capabilities(
• Flexible(
• Knowledge(of(local(
needs/assets(
• Connected(with(
community(members(
• Continued(local(
presence(
Response(
NGOs(Quickly(
Increase(Scope(
and(Scale(of(
Routine(
Responsibilities(
to(Meet(Disaster(
Needs(
(
Recovery(
NGOs(Assume(
Responsibility(for(
Transitioning(
Disaster(Services(
Back(to(Routine(
(
Sustainable&community&development&
Legend:((
((((((((((Components(of(the(model(that(should(be(considered(in(planning(((((((((((((((((Government(or(nongovernment(actor(
((((((((((A(one8way(or(bidirectional(relationship((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((Emergency(event(or(disaster(
((((((((((Point(in(time(when(emergency(event(or(disaster(occurs(
Basedonthesefindingsandtherecommendationsofkeyinformants,weidentifiedsomepriority
elementsthatshouldbeincludedinthetoolkit.TheseelementswillsupportthefacilitatorsofNGO
involvementandaddresschallengesidentifiedthroughtheliteraturereviewandinterviews/focus
groups.Thepriorityelementsareasfollows:
• ReflectiononthehistoricalcontextofNGOengagementduringdisaster.Keyinformants
discussedhowhistoricalcontextinfluencescredibilityandpotentialcontributionsofNGOs.In
ordertofullyunderstandthefactorsthatinfluenceanNGO’scurrentabilitytocontributetoa
community’sdisasterresponseandrecoveryoperations,plannersandNGOsmustfirstreflect
onhistoricalinvolvementandtheimplicationsofpastsuccessesandfailures.Thisincludesa
discussionofpastpartnerships,aswellasthesocial,cultural,andpoliticalcontextofthe
community.
• BetterdefinitionsoftherolesandresponsibilitiesofNGOs.Boththeliteratureandkey
informantsdescribedthechallengescausedbyNGOs’poorlydefinedrolesandresponsibilities
andunderscoredtheneedforatooltohelpNGOs(1)betterarticulatetheircoreserviceareas,
aswellasthekeygovernmentandNGOpartnerstheywillrelyon(orthatrelyonthem)duringa
6
disaster,and(2)betterintegratetheserolesandresponsibilitiesintoorganizationaland
communityplans.
• BetteridentificationoftheassetsthatNGOsbringtodisasterresponseandrecoveryoperations
andimprovedcommunicationofthosetootherNGOsandgovernmentpartners.NGOshave
providedkeyservices,competencies(i.e.,knowledge,skills,expertise),resources,equipment,
anddataandactasabrokerforkeyrelationshipsneededfordisasterresponseandrecovery
(Acosta,Chandra,&Ringel,2013).Boththeliteratureandkeyinformantsindicatedaneedfor
organizationstoimprovetheirunderstandingoftheassetstheybringtothetable,andensure
thattheyarereflectedinorganizationalandcommunityplans.
• ImprovedcommunicationbetweenanNGO(anditsemployeesandconstituents)andotherNGO
andgovernmentpartners.Boththeliteratureandkeyinformantsidentifiedthebreakdownof
communicationbothbeforeandduringdisasterresponseandrecoveryoperationsasamajor
barriertoNGOinvolvement.Toaddressthisbarrier,keyinformantssuggestedidentifyingkey
communicationchannels,developingasystematicprocessforcommunicatingwithpartnersand
trackingemployees/constituents,andintegratingthesecomponentsintoorganizationaland
communityplans.
• Moreemphasisonthelong-termcommitmentofNGOs.Giventhelengthandcomplexityof
disasters,keyinformantsrecommendedthatthetoolhelpNGOsthinkcriticallyabouthow
disasterrecoveryoperationsmayoverlapwithorinfluenceroutineservices,wherethereare
opportunitiesforfundinglong-termservicesandsupports,andwhattheirbenchmarksfor
successoverthelongtermmaybe.
Weusedthesefindingstomapoutcorefeaturesofthetoolkitandcreateafirstdraft.Thosecore
featuresinclude:
• anorganizationalprofileforNGOstoidentifytheirfocus,scope,andcontactinformation
• aself-assessmentofskillsneededfordisasterengagementfocusedonrolesandresponsibilities,
assets,communication,andlong-termrecovery
• anassessmentofthehistoricalcontextforNGOengagement
• animprovement-planningtemplatefororganizationstoimprovetheirskillsandforNGO
networkstoimprovetheirpartnershipswitheachotherandwithgovernmentorganizations.
U.S.PilotTestoftheToolkit
Oncethedrafttoolkitwasdeveloped,weengagedtwocommunitieswithrecentdisasterexperienceand
interestinimprovingNGOengagementindisastertopilottestandassessthetoolkit(Albuquerque,New
Mexico,n=23;Miami,Florida,n=20).Ineachcommunity,weheldaworkshopwithcommunityleaders
representingthediversityofNGOs,includingthosethathadexperiencewithdisasterresponse(e.g.,fire
departments,forestryorganizations,universities)andthosewithexperiencesupportingsocial,
economic,ecological,orotherhumanservices(e.g.,creditunions,soupkitchens,humanesocieties).
Participantswereledthroughaspectsofthedrafttoolkittotesthoweachcomponentworkedoutina
practicalorappliedsettingwithrealusers,andthenhowthetoolkitcouldbeusedasawhole.
Theworkshopstookthreehoursandcovered(1)anoverviewofpurposeandcontentsofthedraft
ENGAGEDtoolkitand(2)ascenario-baseddiscussiononthedisastercontextofeachpilotcommunity
(seetheboxbelowforanexample).Thegoalofthescenarioistotestthestrengthofrelationships
amongNGOsandbetweenNGOsandgovernmentpartners.
7
ExampleScenariofromtheNewMexicoWorkshop
NorthernNewMexicoisexperiencingahistoricallysignificantperiodofdrought,increasingtheriskof
wildfires.Therearemultipleadditionalrisksthataccompanywildfiresoccurringinareassurrounding
SantaClaraPueblo,includingthreatstotheLosAlamosNationalLaboratory(whereresearchonnuclear
energyisconducted)anddestructiveflashfloodinginburnedregions.
AfirestartedintheSantaFeNationalForestyesterday,andsuchweatherconditionsasstrong
unpredictablewindsandlow(orno)precipitationarelikelytocausethefiretospread.Your
organizationshouldinitiateemergencyprotocolstoprepareforpossiblewildfiresandotherrelated
disasters.Theprotocolsaddressnotonlyyourstaff’sactivities,butalsoanychangesorexpansionsto
servicesforyourservicepopulationorconstituents.
Specifically,theworkshopagendaincludedthefollowing:
• PilottestofToolkitSectionOne:GeneralInformationProfile.Weallowedparticipantstotaketen
minutestocompletethegeneralinformationprofileandthenengagedinanexercisetohave
participantsshareoneortwothingsabouttheirorganizationsthattheythinkwouldsurprise
othersintheroom.
• PilottestofToolkitSectionTwo:DisasterSkillsSelf-Assessment.Weaskedparticipantstoreview
theinstructions,askanyquestions,andthencompletetheself-assessmentsectionofthe
toolkit.Onceparticipantshadcompletedtheself-assessment,wewentaroundtheroomand
askedparticipantstoshareonestrengthandoneweaknessfortheirorganizations.
• PilottestofToolkitSectionThree:AssessingPastPartnerships.Wefacilitatedagroupdiscussion
andratingofeachdomain(abouttenminutesforeachofsixdomains).
• PilottestofToolkitSectionFour:NextSteps.Wethenopeneditupfordiscussiontoidentify,
givenpasthistory,whatchangesareneededtoimprovepartnerships.Wefacilitateda
discussiontoidentifybetweenthreeandfiveconcreteactivitiestoimprovepartnershipsover
thenextsixmonths.Wethensuggestedthatparticipantsengageinanotherconversationlike
thisinsixmonthstodeterminetheirprogressandpromotesustainabilityoftheir
improvements.
• Scenario-baseddiscussion.Oncethetoolkitpilottestwascomplete,weaskedparticipantsto
reflectontheoriginalscenariotodeterminewhatwaslearnedfromthetoolkitthatisrelevant
tothescenario.
• Wrap-up.Weadministeredashortsurveytoparticipantstoassesstheirsatisfactionwiththe
workshopandtoolkitmaterialandthankedthemfortheirparticipation.
Findingsfromtheparticipantsurveysuggestedthatparticipantsweresatisfiedwiththeworkshop
(Table1)andfoundthetoolkitusefulorveryuseful(Table2).Ninety-sevenpercentofparticipants
indicatedthattheywouldbelikelyorverylikelytousetheinformationprovidedintheworkshopto
formnewpartnershipsorimprovecoordinationwithexistingpartnersanddevelopneworrevisedplans
fortheirorganization.Whenaskedaboutthesectionsofthetoolkitthattheylikedmost,participants
indicatedthat“allsectionswereveryhelpful”andspecifiedtheplanningtemplateandself-assessment
gridsasparticularlyhelpful.Whenaskedwhichsectionsofthetoolkittheylikedleast,mostparticipants
8
indicated“none”or“N/A”orsuggestedthatitshouldbemoreinteractive.Whenaskedwhatthey
learnedfromtheworkshop,participantsindicatedthattheylearned:
• theimportanceofbuildingpartnershipsandalliancestoimprovetheireffectivenessindisaster
responseandrecovery
• whyitisnecessarytounderstandthemission,roles,andresponsibilitiesofotherlocalpartner
organizations
• thatmanyorganizationsdidnothaveasufficientdisasterplaninplace.
Despiteeachpilottestusingadifferentscenariotailoredtoitsgeographiclocation,thefeedbackonthe
toolkitwassimilarinbothpilottestsinthatparticipantsfromeachlocationindicatedthatthetoolkit
washelpfulandsuggestedmoreinteractivitytoimproveitsuser-friendliness.
Table1.ParticipantFeedbackontheWorkshop(n=37)
Very
Dissatisfied
Somewhat
Dissatisfied
Somewhat
Satisfied
Very
Satisfied
a.Qualityofthe
informationprovidedduring
today’sworkshop
8%
0
24%
68%
b.Relevanceofthe
informationtoyourwork
8%
0
22%
70%
c.Clarityoftheworkshop
8%
0
22%
70%
Table2.ParticipantFeedbackonToolkitSections
Very
Useful
Somewhat
Useful
Somewhat
Useful
Very
Useful
3%
3%
28%
66%
3%
3%
25%
69%
3%
6%
31%
60%
d.Creatinganimprovementplan
(n=36)
3%
3%
22%
72%
e.Disasterscenarioexercise
(n=34)
0%
6%
32%
62%
a.Completingandsharinggeneral
informationprofiles(n=36)
b.Disasterskillsself-assessment
(n=36)
c.Assessingpastpartnerships
(n=35)
Inresponsetoparticipants’feedback,weconvertedpartsofthepaper-basedversionofthetoolkitinto
anonlineinteractivetool.SectionsTwoandThreeofthetoolkitareavailableinaninteractiveformatat
www.rand.org/pubs/tools/TL202.html.
Thegoalofthesesectionsistoprovideeasy-to-use,onlinechecklistswithouthavingtheuserwork
throughtheentiretoolkit.
9
Duringpilottesting,wefieldedseveralquestionsabouthowtoimplementthetoolkit.Asaresultof
input,weaddedsometexttotheintroductionofthetoolkitthatinstructsusershowthetoolkitcanbe
completedeither(1)independentlybyasingleNGOtobenchmarkitsorganizationalcapabilitiesto
engageduringadisasteror(2)aspartofabroaderemergencyplanninggroup(e.g.,assembleagroupof
NGOs,thenwalkthroughthetoolkitsections).Wealsoaddedtexttotheintroductiondescribingthe
sectionsofthetoolkitandassociatedexercisesinmoredetailtohelpusersnavigatethecontent.
NextSteps
Tofurthertesttheoperationalmodelthatthetoolkitisbasedon,weconductedasurveyof226NGOs
thatassessestheircapabilities,services,andperceivedengagementinandeffectondisasterresponse
andrecoveryintheircommunities.WerecruitedNGOsforthesurveyfrommemberlistsforthestate
chaptersofVOADs.Weaskedrespondentstocompletethesurveyonbehalfoftheirentireorganization.
Respondentswereofferedtheoptiontoreceivea$20giftcardtoreimbursethemforthetimespent
completingthesurvey.Wearecurrentlyanalyzingthesurveydataandwritingmanuscriptstoshare
surveyfindings.Notethatthefindingsareintendednottoinformthetoolkitbuttoprovideauseful
complementtoouranalysesanddevelopmentmodel.
References
Acosta,J.,&Chandra,A.(2013).Harnessingcommunityforsustainabledisasterresponseandrecovery:
Anoperationalmodelforintegratingnongovernmentalorganizations.DisasterMedicineand
PublicHealthPreparedness,7(4):361–368.
Acosta,J.,Chandra,A.,&Ringel,J.(2013).Nongovernmentalresourcestosupportdisaster
preparedness,response,andrecovery.DisasterMedicineandPublicHealthPreparedness,7(4):
348–353.
Acosta,J.,Chandra,A.,Sleeper,S.,&Springgate,B.(2011).Thenongovernmentalsectorindisaster
resilience:Conferencerecommendationsforapolicyagenda.SantaMonica,CA:RAND
Corporation,CF-282-ALLF.AsofMarch21,2016:
http://www.rand.org/pubs/conf_proceedings/CF282.html
Chandra,A.,&Acosta,J.(2009).Theroleofnongovernmentalorganizationsinlong-termhuman
recoveryafterdisaster:ReflectionsfromLouisianafouryearsafterHurricaneKatrina.Santa
Monica,CA:RANDCorporation,OP-277-RC.AsofMarch21,2016:
http://www.rand.org/pubs/occasional_papers/OP277.html
Cutter,SusanL.,Emrich,C.T.,Mitchell,J.T.,Boruff,B.J.,Gall,M.,Schmidtlein,M.C.,Burton,C.G.,&
Melton,G.(2006,March).Thelongroadhome:Race,class,andrecoveryfromHurricane
Katrina.Environment:ScienceandPolicyforSustainableDevelopment,48(2):8–20.
Moore,M.,Chandra,A.,&Feeney,K.(2013).Buildingcommunityresilience:WhattheUnitedStatescan
learnfromexperiencesinothercountries?DisasterMedicineandPublicHealthPreparedness
7(3):292–301.
Waugh,WilliamL.,Jr.,(2006,March).ThepoliticalcostsoffailureintheKatrinaandRitaDisasters.
AnnalsoftheAmericanSocietyofPoliticalandSocialScience,604(1):10–25.
10