Disaster Preparedness Calcasieu Parish Public Schools Sheryl Abshire Administrative Coordinator of Technology Lake Charles, Louisiana Disaster Background • Hurricane Rita struck the Louisiana / Texas border on September 24, 2005 as a category 3 storm with 120 mph sustained winds • Calcasieu Parish was hit by the hurricane eyewall and the east quadrant which has the strongest winds • Many schools in Calcasieu Parish received extensive roof and water damage. The lack of power afterwards promoted mold & mildew growth • 24 hours after Rita hit, the CPSB web & email servers were back up and providing information to evacuees across the country • 34 days later, CPSB schools reopened Top 10 Hurricane Tips • 10 – After eating MREs for a week, frozen burritos CAN be a real treat • 9 – It takes at least 300 “C” batteries to power-up a server • 8 - Make sure the fish on your computer screen are from a screen saver and not real fish • 7 - Food packets are more important than data packets after the storm Top 10 Hurricane Tips • 6 – Having a tree in your server room doesn’t necessarily mean it’s Christmas • 5 - Help raise recovery funds by starting a lottery to use the one working computer to check email • 4 – Most chainsaws & generators do not have USB or Ethernet ports Top 10 Hurricane Tips • 3 – Although getting back to nature is good, mold & mildew are poor electrical conductors inside computers • 2 – Consider pitching a new show to the networks called Extreme Makeover: Server Room • 1 – After a hurricane, dial-up Internet IS considered broadband access Preparing for the Unexpected • A plan needs to exist before its needed; making one on the fly is too late • Know your mission critical operations • Play out scenarios – see how the plan can fit different situations • Think creatively • Pay attention to details Preparing for the Unexpected • Strategic Purchasing Decisions – Natural Gas Powered Generator to maintain essential systems – Use of rack mounted servers to systems can be moved out in case of disaster – Standardization of equipment and services – Redundant backup systems Communication is critical • Communicate to district staff what needs to be done and when • What information will need to go out after an event • How will information be distributed to staff & stakeholders • Have emergency contact numbers & check-in for staff – they could wind up anywhere in the U.S. Tech Staff Evacuation Locations Staged Shutdown • Develop a staged shutdown that moves in steps from simple preparedness to ceasing operations • Protect assets while staff is available to do the work • Shutdown must be staged so that mission critical operations are the last to be stopped • Shutdown should be designed so that it can be reversed if needed Example – 72 Hours Before • Email staff to power down computers at the end of the day • Backup any critical data on personal computers • Unplug computers from electricity • All of these are easily reversed if not needed Stage One Email Example – 48 Hours Before • All steps from earlier stage • Move critical servers to protected areas • Move computers off of floor and cover to protect from water • Backup of mission critical data is performed • Informed service providers and key technology providers of planned shut down and needed assistance Stage Two Email Example – 24 Hours Before • All critical data center servers moved out of impact area • All network infrastructure is shut down • Data backups are sent out to areas outside of the impact area • Arrange for alternate data center locations Lessons Learned….Moving Forward • Public relations requires a single point of contact • Mis-information will kill you. You must have a consistent delivery of information across multiple channels….web, email, press, toll free number • Community-wide partnerships are essential – especially your communications providers, community leaders & first responders • There’s no room for egos Lessons Learned….Moving Forward • Scale to the worst case potential disaster • Update your business continuance plan • Evaluate effectiveness of processes…use common sense • Payroll is THE mission critical system • Electronic deposit should be mandatory • Pre-determine and connect with necessary national vendors to restore critical systems Lessons Learned….Moving Forward • Coordinating restoration of services with local authorities…CPSB badges, credentials, pre-determined authorized accesses, etc. • Stockpile survival necessities – bottled water, flashlights, canned foods, etc. • Plans for deliveries after disaster Lessons Learned….Moving Forward • Plan for evaluating, repairing damaged equipment • Planning for facilities…example doorway sizes to roll out server racks • Redundant back-ups in other off site locations • Set goal to move a data systems out within a 4 hour period • Plan for systematically bringing data services back on line • Key staff with strong leadership and the ability to think creatively is critical to success CONTACT INFORMATION Calcasieu Parish Public Schools Lake Charles, Louisiana http://www.cpsb.org • Sheryl Abshire Administrative Coordinator of Technology sheryl.abshire@cpsb.org • Scott Schonefeld Tech Support Center Coordinator scott.schonefeld@cpsb.org http://www.cpsb.org/presentations