FDOT Safety Office: Crash Data Management Crash Data Management Florida Traffic Safety Portal Crash Data Management: Crash Data Sources Crash Data Flow: o Florida Traffic Crash Report (reporting officer) o Preliminary Processing (reporting agency) o Secondary Processing/Validation/Custodian (DHSMV) o Transfer of Long Form Data (SSO) FDOT SSO Crash Location Processing: o Upload and Conversion o Location Processing and QA/QC Location Processing on the SHS Location Processing for Local Roads FDOT SSO Reports and Analyses o The Crash Analysis Reporting (CAR) System o Crash Location Shapefiles Crash Data Sources By law, the Florida Department of Highway Safety and Motor Vehicles (DHSMV) is the official custodian of the crash reports and is responsible for statewide crash data collection and dissemination of the official statistics. Crash reports that reveal personal information of parties involved in the crash are confidential and exempt from the provisions of §119.07(1), F.S. and s. 24(a), Article 1 of the Constitution for a period of 60 days after the date the report is filed. (§316.066 (2)(a), F.S.). The Florida Department of Transportation (FDOT) State Safety Office (SSO) receives the Long Form crash data from the DHSMV and loads them into the Crash Analysis Reporting (CAR) database. As of January, 2014, approximately 65% of all documented crashes were submitted to DHSMV using the long form. The SSO uses the CAR system to process the crashes and to support safety analyses for Florida’s public roads. Page 1 www2.dot.state.fl.us/trafficsafetywebportal FDOT Safety Office: Crash Data Management Crash Data Flow Florida Traffic Crash Report (reporting officer) Crash incidents are documented by law enforcement agencies using the Florida Traffic Crash Report (FTCR) long form, the FTCR short form, or on a driver-exchange form depending upon the location and severity of the crash. The material difference between the long and short forms is that the long form requires both a narrative and a diagram and short form does not require either. For additional requirements, please refer to the excerpt below from Section 316.066, F.S.: 316.066 Written reports of crashes.— (1)(a) A Florida Traffic Crash Report, Long Form must be completed and submitted to the department within 10 days after an investigation is completed by the law enforcement officer who in the regular course of duty investigates a motor vehicle crash that: 1. Resulted in death of, personal injury to, or any indication of complaints of pain or discomfort by any of the parties or passengers involved in the crash; 2. Involved a violation of s. 316.061(1) or s. 316.193; 3. Rendered a vehicle inoperable to a degree that required a wrecker to remove it from the scene of the crash; or 4. Involved a commercial motor vehicle. (b) The Florida Traffic Crash Report, Long Form must include: 1. The date, time, and location of the crash. 2. A description of the vehicles involved. 3. The names and addresses of the parties involved, including all drivers and passengers, and the identification of the vehicle in which each was a driver or a passenger. 4. The names and addresses of witnesses. 5. The name, badge number, and law enforcement agency of the officer investigating the crash. 6. The names of the insurance companies for the respective parties involved in the crash. (c) In any crash for which a Florida Traffic Crash Report, Long Form is not required by this section and which occurs on the public roadways of this state, the law enforcement officer shall complete a short-form crash report or provide a driver exchange-of-information form, to be completed by all drivers and passengers involved in the crash, which requires the identification of each vehicle that the drivers and passengers were in. The short-form report must include: 1. The date, time, and location of the crash. 2. A description of the vehicles involved. 3. The names and addresses of the parties involved, including all drivers and passengers, and the identification of the vehicle in which each was a driver or a passenger. 4. The names and addresses of witnesses. 5. The name, badge number, and law enforcement agency of the officer investigating the crash. www2.dot.state.fl.us/trafficsafetywebportal Page 2 FDOT Safety Office: Crash Data Management 6. The names of the insurance companies for the respective parties involved in the crash. Preliminary Processing (reporting agency) The long and short form crash reports from law enforcement officers are submitted to supervisors who review and approve the reports for submittal. Upon approval by a supervisor, electronic crash reports are transmitted to the DHSMV through electronic submittal using Extensible Markup Language (XML) via File Transfer Protocol (FTP). Crash reports submitted on paper forms are mailed to the DHSMV to be scanned and converted into Tagged Image File Format (TIFF) and then forwarded to a third party contractor who performs the necessary data entry. Secondary Processing/Validation/Custodian (DHSMV) Once the electronic files have been created, the records are checked by automated routines that look for data errors and discrepancies. All of the records that pass the automated check are uploaded into the DHSMV database within 24 hours of submittal. Electronic records that fail the validation checks are returned to the submitting agency for correction and resubmission. For paper crash reports, the data entry contractor converts the records into electronic format (comma-separated values (CSV)) and transfers them to the DHSMV daily via FTP. Paper crash reports that fail the quality checks are identified and returned to the submitting agency with notes identifying the issues. The responsible agencies must then correct and resubmit the reports. Transfer of Long Form Data (SSO) Every weekday, the DHSMV generates a file of that day’s or weekend’s new long form reported crashes and any updates to existing crashes that have been received, both from direct electronic submittal from the reporting agencies and the data entry contractor. The crash data file is in CSV format. The DHSMV combines the CSV file with the corresponding crash image files then encrypts the combined file and posts it for the SSO to download. The SSO transfers and unpacks the file and stores the record data in CAR, a DB2 database, and the corresponding TIFF images in an Electronic Document Management System (EDMS). Updated records match to and overwrite the original database records; additionally, key report fields are flagged for duplicates and resolved through DHSMV’s reciprocal process. Finally, the incoming TIFF files for updated records are added to the EDMS database with a version number. Page 3 www2.dot.state.fl.us/trafficsafetywebportal FDOT Safety Office: Crash Data Management FDOT SSO Crash Location Processing Upload and Conversion After the crash data is downloaded from DHSMV, the CAR System's automated process checks and formats them for upload into the CAR database. The records are split into groupings to match the CAR database structure and certain fields are calculated and flagged. DOT is working through a challenge created when the FTCR forms were modified in 2011. DHSMV instituted this change to align Florida crash data collection protocols closer with the Federal Model Uniform Crash Codes (MUCC). At the current time, the CAR database has not been updated to store all of the new fields on the updated form. However, the incoming 2011 crash data are converted as accurately as possible to match the previous FTCR data elements and code values uploaded into the CAR database. Location Processing and QA/QC Using DHSMV crash data and crash report images in combination with other FDOT data, the SSO Crash Records and Research Section processes crash records to determine an exact location for each crash. Once the crashes have location coordinates, the SSO systems use them to perform various analyses. Location processing is conducted separately for the State Highway System (SHS) and for Local Roads. Location Processing on the SHS Crashes are first screened by an automated location process within the CAR System. Unambiguous locations on the SHS are captured and spot-checked for quality and accuracy. Records successfully located on the SHS by the automated process, based on a recent sample, make up just over 13 percent of the total crashes received (ranging from 8 percent to 18 percent on any given day). Also captured are records that can be unambiguously confirmed not on the SHS. These records fall into two categories: (1) crashes that are on private property or in parking lots and are not on the public roads system, and (2) crashes for which the automated process can determine that the referenced location is not on the SHS. The former (1) make up approximately 11 percent of the crashes received (also ranging from 8 to 18 percent) and the latter (2), make up approximately 14 percent (ranging from 12 to 16 percent). The remaining records (about 62 percent of the records received) are crashes for which the system cannot automatically determine a location or crashes that involved a fatality. The indeterminate locations and the fatal crashes are reviewed and finalized by personnel in the Crash Records and Research section using the CAR System. Processed SHS locations are reviewed for quality and final approval for inclusion in the reporting and analysis dataset. www2.dot.state.fl.us/trafficsafetywebportal Page 4 FDOT Safety Office: Crash Data Management Location Processing for Local Roads Records for crashes that the SSO staff have determined are not on the SHS are passed to a second system for location processing using customized Geographic Information Systems (GIS) applications. Crash locations that are on Florida public roads but are not on the SHS are imported by the SSO’s Crash Locator System (CLS). In the CLS the records are updated with corrected street names, given latitude and longitude coordinates and linked to a specific roadway segment in the map dataset used to determine the location. The CLS crash locations are verified and quality checked by SSO staff and after initial location. FDOT SSO Reports and Analyses The Crash Analysis Reporting (CAR) System The SSO, through the CAR System, has a powerful and flexible reporting tool that can provide crash and crash analysis data in a variety of formats. The CAR System can provide specific crash data for any location on the SHS and can filter the crashes returned by most of the data elements that are collected on the FTCR Long Form. Standard output formats are live (on-screen), formatted reports in PDF and data files in CSV format. The formatted reports are available as a crash listing with limited attribute data, a section or location crash rate analysis with summary tables for a standard set of data elements and as a customized tabulation on elements selected by the user. The CSV files are available in six versions, which vary based on the attributes included in the extract, or as an extract of all attributes in the database. The CAR System runs annual analyses for crashes on the SHS and the results are available for crash rates and high-crash locations, as well as for areas where certain crash types have a higher than average concentration (night-time crashes, wet weather crashes, et al.). Annual analyses are also generated for the non-SHS public roads using the Crash Locator Analysis Reports (CLAR) System. Results are published on the SSO’s Florida Traffic Safety Portal: http://webapp02.dot.state.fl.us/TrafficSafetyWebPortalFDOT/. Crash Location Shapefiles Because the location coordinates for the crashes that are not on the SHS are not stored in the CAR System, the CAR System has only a limited capability for reporting on crashes that are not on the SHS. The CLS stores the geographic coordinates for the crashes that are on non-SHS public roads, but there is no generl reporting function in the CLS. At the close of each calendar year (and sometimes in intermediate intervals) the location coordinates in the CLS are captured and the SSO generates and publishes crash location shapefiles that allow users to access crashes by location in a GIS format. Page 5 www2.dot.state.fl.us/trafficsafetywebportal