CPSB Disaster Preparedness Presentation

advertisement
Disaster Preparedness
Calcasieu Parish Public Schools
Sheryl Abshire
Administrative
Coordinator
of Technology
Lake Charles,
Louisiana
Disaster Background
• Hurricane Rita struck the Louisiana / Texas border on
September 24, 2005 as a category 3 storm with 120 mph
sustained winds
• Calcasieu Parish was hit by the hurricane eyewall and the
east quadrant which has the strongest winds
• Many schools in Calcasieu Parish received extensive roof
and water damage. The lack of power afterwards
promoted mold & mildew growth
• 24 hours after Rita hit, the CPSB web
& email servers were back up and
providing information to evacuees
across the country
• 34 days later, CPSB schools reopened
Top 10 Hurricane Tips
• 10 – After eating MREs for a week, frozen
burritos CAN be a real treat
• 9 – It takes at least 300 “C” batteries to power-up
a server
• 8 - Make sure the fish on your computer screen
are from a screen saver and not real fish
• 7 - Food packets are more important than data
packets after the storm
Top 10 Hurricane Tips
• 6 – Having a tree in your server room
doesn’t necessarily mean it’s Christmas
• 5 - Help raise recovery funds by starting a
lottery to use the one working computer to
check email
• 4 – Most chainsaws & generators do not
have USB or Ethernet ports
Top 10 Hurricane Tips
• 3 – Although getting back to nature is good,
mold & mildew are poor electrical
conductors inside computers
• 2 – Consider pitching a new show to the
networks called Extreme Makeover: Server
Room
• 1 – After a hurricane, dial-up Internet IS
considered broadband access
Preparing for the Unexpected
• A plan needs to exist before its needed;
making one on the fly is too late
• Know your mission critical operations
• Play out scenarios – see how the plan can
fit different situations
• Think creatively
• Pay attention to details
Preparing for the Unexpected
• Strategic Purchasing Decisions
– Natural Gas Powered Generator to maintain
essential systems
– Use of rack mounted servers to systems can
be moved out in case of disaster
– Standardization of equipment and services
– Redundant backup systems
Communication is critical
• Communicate to district staff what needs to be
done and when
• What information will need to go out after an
event
• How will information be distributed to staff &
stakeholders
• Have emergency contact numbers & check-in for
staff – they could wind up anywhere in the U.S.
Tech Staff Evacuation Locations
Staged Shutdown
• Develop a staged shutdown that moves in steps
from simple preparedness to ceasing operations
• Protect assets while staff is available to do the
work
• Shutdown must be staged so that mission critical
operations are the last to be stopped
• Shutdown should be designed so that it can be
reversed if needed
Example – 72 Hours Before
• Email staff to power down computers at the
end of the day
• Backup any critical data on personal
computers
• Unplug computers from electricity
• All of these are easily reversed if not
needed
Stage One Email
Example – 48 Hours Before
• All steps from earlier stage
• Move critical servers to protected areas
• Move computers off of floor and cover to
protect from water
• Backup of mission critical data is performed
• Informed service providers and key
technology providers of planned shut down
and needed assistance
Stage Two Email
Example – 24 Hours Before
• All critical data center servers moved out of
impact area
• All network infrastructure is shut down
• Data backups are sent out to areas outside
of the impact area
• Arrange for alternate data center locations
Lessons Learned….Moving
Forward
• Public relations requires a single point of contact
• Mis-information will kill you. You must have a
consistent delivery of information across multiple
channels….web, email, press, toll free number
• Community-wide partnerships are essential –
especially your communications providers,
community leaders & first responders
• There’s no room for egos
Lessons Learned….Moving
Forward
• Scale to the worst case potential disaster
• Update your business continuance plan
• Evaluate effectiveness of processes…use
common sense
• Payroll is THE mission critical system
• Electronic deposit should be mandatory
• Pre-determine and connect with necessary
national vendors to restore critical systems
Lessons Learned….Moving
Forward
• Coordinating restoration of services with
local authorities…CPSB badges,
credentials, pre-determined authorized
accesses, etc.
• Stockpile survival necessities – bottled
water, flashlights, canned foods, etc.
• Plans for deliveries after disaster
Lessons Learned….Moving
Forward
• Plan for evaluating, repairing damaged equipment
• Planning for facilities…example doorway sizes to roll out
server racks
• Redundant back-ups in other off site locations
• Set goal to move a data systems out within a 4 hour
period
• Plan for systematically bringing data services back on line
• Key staff with strong leadership and the ability to think
creatively is critical to success
CONTACT INFORMATION
Calcasieu Parish Public Schools
Lake Charles, Louisiana
http://www.cpsb.org
• Sheryl Abshire
Administrative Coordinator of Technology
sheryl.abshire@cpsb.org
• Scott Schonefeld
Tech Support Center Coordinator
scott.schonefeld@cpsb.org
http://www.cpsb.org/presentations
Download