Automated Cyclone Discovery and Tracking using Knowledge Sharing in Multiple Heterogeneous Satellite Data Authors Shen-Shyang Ho Ashit Talukder Jet Propulsion Laboratory California Institute of Technology Group 3 Karen Simpson Paul Fomenky Roman Sizov Sameh Ebeid Assignment 1 02/22/2010 Outline 2 Introduction Previous Work Data Description Issues and Challenges Heterogeneous Remote Satellite-Based Detection and Tracking Approach Experimental Results Lessons Learned and Conclusions 2/22/2010 Automated Cyclone Discovery and tracking Introduction What is Cyclone 3 An area of closed, circular fluid motion rotating in the same direction as the Earth Low pressure areas, their center is the lowest atmospheric pressure in the region 2/22/2010 Automated Cyclone Discovery and tracking Introduction Surface-based Types 4 Polar cyclone Polar low Extra-tropical Sub-tropical Tropical Mesoscale 2/22/2010 Automated Cyclone Discovery and tracking Introduction Extra-tropical 5 Synoptic scale low pressure weather system that has neither tropical nor polar characteristics Often described as depressions or lows by weather forecasters 2/22/2010 Automated Cyclone Discovery and tracking Introduction Tropical 6 Storm characterized by a low pressure center and numerous thunderstorms that produce strong winds and flooding rain Referred to by other names such as hurricane, typhoon, tropical storm Develop over large bodies of warm water, and lose strength if they move over land 2/22/2010 Automated Cyclone Discovery and tracking Introduction Tropical 7 An average 86 tropical cyclones of tropical storm intensity form annually worldwide, 47 reaching hurricane/typhoon strength, and 20 becoming intense tropical cyclones 2/22/2010 Automated Cyclone Discovery and tracking 8 2/22/2010 Automated Cyclone Discovery and tracking Introduction Cyclone detection and tracking 9 The tropical prediction center / National Hurricane Center (TPC/NHC) use conventional surface and upper-air observations and reconnaissance aircraft report In recent years, some studies have used satellite images that are manually retrieved and analyzed to improve the accuracy of cyclone tracking 2/22/2010 Automated Cyclone Discovery and tracking Introduction Cyclone detection and tracking A new automated global cyclone discovery and tracking approach on a truly global basis using near real-time (NRT) and historical sensor data from multiple satellite This implementation employs two types of satellite sensor measurements – – 10 QuikSCAT wind satellite data Merged precipitation data using TRMM and other satellites 2/22/2010 Automated Cyclone Discovery and tracking Introduction Cyclone detection and tracking Challenges pertaining to mining data from orbiting satellites – 11 Each orbiting satellite cannot monitor a region continuously and the measurements are instantaneous 2/22/2010 Automated Cyclone Discovery and tracking Introduction Cyclone detection and tracking Challenges pertaining to mining data from orbiting satellitesCan minimize their effects by – 12 using data from multiple satellite Each orbiting satellite cannot monitor a region continuously and the measurements are instantaneous 2/22/2010 Automated Cyclone Discovery and tracking Introduction Cyclone detection and tracking Challenges pertaining to mining data from orbiting satellites – – – 13 Each orbiting satellite cannot monitor a region continuously and the measurements are instantaneous Different satellites provide different measurements Different satellites sensors acquire measurements at different spatial and temporal resolution 2/22/2010 Automated Cyclone Discovery and tracking Introduction Cyclone detection and tracking Challenges pertaining to mining These problems makedata from orbiting satellites mining heterogeneous data – – – 14 from multiple orbiting Each orbiting satellite cannot monitor a region satellites extremely are continuously and the measurements challenging and remains instantaneous as a provide now primarily an measurements Different satellites different unsolved problem Different satellites sensors acquire measurements at different spatial and temporal resolution 2/22/2010 Automated Cyclone Discovery and tracking Introduction Cyclone detection and tracking Challenges related to the problem of detection and tracking of cyclones – – – 15 Cyclone events are dynamic in nature There is lack of annotated negative (non-cyclone) examples by experts A single satellite sensor may miss a cyclone event due to a pre-defined orbiting trajectory 2/22/2010 Automated Cyclone Discovery and tracking Outline 16 Introduction Previous Work Data Description Issues and Challenges Heterogeneous Remote Satellite-Based Detection and Tracking Approach Experimental Results Lessons Learned and Conclusions 2/22/2010 Automated Cyclone Discovery and tracking Previous work Previous work 17 No solution currently exists that uses heterogeneous sensor measurement to automatically detect and track cyclones The current solutions involve human interference and decision 2/22/2010 Automated Cyclone Discovery and tracking Outline 18 Introduction Previous Work Data Description Issues and Challenges Heterogeneous Remote Satellite-Based Detection and Tracking Approach Experimental Results Lessons Learned and Conclusions 2/22/2010 Automated Cyclone Discovery and tracking Data description QuikSCAT Wind Data 19 The QuikSCAT (Quick Scatterometer) mission provide important high quality ocean wind data set Recent research showed QuikSCAT data is useful for early detection of tropical cyclones 2/22/2010 Automated Cyclone Discovery and tracking Data description Precipitation Data from TRMM satellite 20 The Tropical Rainfall Measurement Mission (TRMM) is a joint mission between NASA and JAXA designed to monitor and study tropical rainfall The (Level) 3b-42 TRMM data product used in this paper is produced using a combined instrument rain calibration algorithm 2/22/2010 Automated Cyclone Discovery and tracking Outline 21 Introduction Previous Work Data Description Issues and Challenges Heterogeneous Remote Satellite-Based Detection and Tracking Approach Experimental Results and Conclusions 2/22/2010 Automated Cyclone Discovery and tracking Issues and Challenges 22 Main issues and challenges Non-Continuous Region Monitoring Event Occlusion Varying Temporal and Spatial Resolution Lack of Annotated Negative Examples 2/22/2010 Automated Cyclone Discovery and tracking Main issues and challenges 23 Satellite measurements are instantaneous; hence, satellites cannot measure sustained winds. Remember, a leading characteristic of cyclones is sustained winds TRMM 3B42 data is known to underestimate rainfall, which might lead to false negatives 2/22/2010 Automated Cyclone Discovery and tracking Non-Continuous Region Monitoring – Problem 24 Geostationary Operational Environmental Satellites (GOES) monitor specific area at all times, helping identify “sustained” winds etc. Unfortunately, most countries do not have these. Because QuikSCAT and TRMM are motile, this monitoring is “lost.” This results in “invisible” swaths. 2/22/2010 Automated Cyclone Discovery and tracking print Non-Continuous Region Monitoring – Problem Evidence 25 2/22/2010 Automated Cyclone Discovery and tracking Non-Continuous Region Monitoring – Operational Weather Satellite System Satellite systems consist of two types – – Geostationary Operational Environmental Satellites are static and throw light on current and short term weather trends. Orbiting satellites like QuikSCAT and TRMM help with longer term forecasting. http://noaasis.noaa.gov/NOAASIS/ml/genlsatl.html 26 Automated Cyclone Discovery and tracking Non-Continuous Region Monitoring – Solution 27 Usage of multiple satellites produces a higher temporal density hence helping alleviate the problem. A group of complementary satellites can make this problem almost insignificant. 2/22/2010 Automated Cyclone Discovery and tracking Event Occlusion - Problem 28 Satellite swath can partially (or worst case, totally) miss events of interest. Though in continuous orbit, event can be gone by time satellite comes back. 2/22/2010 Automated Cyclone Discovery and tracking print Event Occlusion – Problem Evidence 1 29 QuikSCAT showing only a small part of event of interest. Hurricane Dean – Aug 17th 2007, 0900 2/22/2010 Automated Cyclone Discovery and tracking print Event Occlusion – Problem Evidence 2 30 Next QuikSCAT swath shows a bit more. Hurricane Dean – Aug 17th 2007, 1041 2/22/2010 Automated Cyclone Discovery and tracking print Event Occlusion – Problem Evidence 3 31 Another QuikSCAT swath shows much more, but missing eye of storm. Hurricane Dean – Aug 17th 2007, 2310 2/22/2010 Automated Cyclone Discovery and tracking print Event Occlusion – Problem Evidence 4 32 QuikSCAT swath from previous day showed more! Hurricane Dean – Aug 16hth 2007, 2156 2/22/2010 Automated Cyclone Discovery and tracking Event Occlusion – Solution 33 Clearly, multiple orbits of the same satellite can produce more information on the event being examined. Also, as in continuity monitoring issue, numerous satellites working together are less likely to miss important events. 2/22/2010 Automated Cyclone Discovery and tracking Varying Temporal and Spatial Resolution – Problem Different aspects influence the temporal resolution of measurements: – – – 34 Satellite orbit time (QuikSCAT 101 minutes, TRMM 92.5mins) Swath width of measuring instrument (SeaWinds on QuikSCAT 1800km; PR, TMI and VIRS on TRMM 247km, 878km, 873km respectively) Geographic coverage (QuikSCAT – global; TRMM – 50N to 50S) 2/22/2010 Automated Cyclone Discovery and tracking Varying Temporal and Spatial Resolution – Problem Cont’d Spatial resolution depends on – – – 35 Sensor instruments (PR, TMI and VIRS on TRMM 5.1km, 5.0km, 2.4km respectively) Satellite orbital altitude ((TRMM Pre-boost (350km) (TMI): 4.4km to 5.1km (Post-boost (403 km)) Processing algorithm (operational QuikSCAT data has spatial resolutions of 12.5km and 25km ) 2/22/2010 Automated Cyclone Discovery and tracking Varying Temporal and Spatial Resolution – Problem Cont’d 2 In addition to inter satellite differences, there are some intra satellite tempo-spatial differences. – 36 TRMM Level 3 data has lower temporal resolution than levels 1 and 2. 2/22/2010 Automated Cyclone Discovery and tracking Varying Temporal and Spatial Resolution – Solution 37 On TRMM, mine areas QuikSCAT showed events of interest on. Also, because of different swath sizes, latitudes and longitudes were used to identify locations. Temporal tracking done on TRMM as temporal resolution higher than in QuikSCAT. 2/22/2010 Automated Cyclone Discovery and tracking Lack of Annotated Negative examples - Problem 38 Scientists have not clearly shown what a “non-event” is despite the large archives of events. 2/22/2010 Automated Cyclone Discovery and tracking Lack of Annotated Negative examples - Solution 39 Random “non-event” days were monitored and fed to system as examples of non event. 2/22/2010 Automated Cyclone Discovery and tracking 40 Introduction Previous Work Data Description Issues and Challenges Heterogeneous Remote Satellite-Based Detection and Tracking Approach Experimental Results and Conclusions 2/22/2010 Automated Cyclone Discovery and tracking Heterogeneous Remote Satellite-Based Detection and Tracking Approach Heterogeneous Remote Satellite-Based Detection and Tracking Approach QuikSCAT Feature Selection Ensemble Classifier for Cyclone Detection Knowledge Sharing between TRMM and QuikSCAT data for Cyclone Tracking 41 2/22/2010 Automated Cyclone Discovery and tracking Heterogeneous Remote Satellite-Based Detection and Tracking Approach QuikSCAT Feature Selection 42 Features that characterize and identify a cyclone are selected from QuikSCAT satellite data The QuikSCAT Level 2B data that consist of ocean wind vector information are utilized The Level 2B data are grouped by rows of wind vector cells (WVC) which are squares of dimension 25 km or 12.5 km 2/22/2010 Automated Cyclone Discovery and tracking Heterogeneous Remote Satellite-Based Detection and Tracking Approach QuikSCAT Feature Selection (cont`d) 43 1624 WVC rows at 25 km or 3248WVC rows at 12.5 are required to cover the earth circumference Out of 25 fields in the data structure for the Level 2B data we are interested only in latitude, longitude, wind speed(WS) and wind direction (WD) 2/22/2010 Automated Cyclone Discovery and tracking Heterogeneous Remote Satellite-Based Detection and Tracking Approach QuikSCAT Feature Selection (cont`d) Table 1. The fields of interest from Level 2B data structure 44 Field Unit Minimum Maximum WVC latitude Deg -90.00 90.00 WVC longitude Deg E 0.00 359.99 Selected speed m/s 0.00 50.00 Selected direction Deg from North 0.00 359.99 2/22/2010 Automated Cyclone Discovery and tracking Heterogeneous Remote Satellite-Based Detection and Tracking Approach QuikSCAT Feature Selection (cont`d) 45 The Level 2B data needs to be interpolated on a uniformly gridded surface due to the non-uniformity in the measurements taken by the QuikSCAT satellite on a spherical surface The nearest neighbor rule is used for this pre-processing procedure for both wind speed (WS) and wind direction (WD) 2/22/2010 Automated Cyclone Discovery and tracking Heterogeneous Remote Satellite-Based Detection and Tracking Approach QuikSCAT Feature Selection (cont`d) 46 Histograms are constructed to estimate probability density of the wind speed (WS) and wind direction (WD) within a predefined bounding box extracted from a QuikSCAT image 2/22/2010 Automated Cyclone Discovery and tracking Heterogeneous Remote Satellite-Based Detection and Tracking Approach QuikSCAT Feature Selection (cont`d) 47 WS(i,j),WD(i,j) – wind speed and wind direction at location (i,j) DSR(i,j) – the direction to speed ratio at (i,j) 2/22/2010 Automated Cyclone Discovery and tracking Heterogeneous Remote Satellite-Based Detection and Tracking Approach QuikSCAT Feature Selection (cont`d) 48 When there is a strong wind with wind circulation, the DSR at a WVC will be small DSR histogram will have a skewed distribution towards the smaller value When there is weak or no wind with no circulation, DSR histogram does not have the skewed characteristics 2/22/2010 Automated Cyclone Discovery and tracking Heterogeneous Remote Satellite-Based Detection and Tracking Approach QuikSCAT Feature Selection (cont`d) 49 When a region contains a cyclone, the WS histogram shows a density estimate skewed towards the larger values and WD histogram shows a “near uniform” distribution A cyclone is defined as a “warm-core nonfrontal synoptic-scale” system, with “organized deep convection and a closed surface wind circulation about a well-defined center” 2/22/2010 Automated Cyclone Discovery and tracking Heterogeneous Remote Satellite-Based Detection and Tracking Approach QuikSCAT Feature Selection (cont`d) 50 To discriminate between cyclone and noncyclone events based on the circulation property two additional features are used: (1) a measure of relative strength of the dominant wind direction (DOWD) (2) the relative wind vorticity (RWV) 2/22/2010 Automated Cyclone Discovery and tracking Heterogeneous Remote Satellite-Based Detection and Tracking Approach QuikSCAT Feature Selection (cont`d) 51 u(i,j) and v(i,j) are the u-v components of the wind direction WD(i,j) at location (i,j) with 1≤i ≤m and 1≤j≤n The (mn)-by-2 matrices M are constructed as follows: 2/22/2010 Automated Cyclone Discovery and tracking Heterogeneous Remote Satellite-Based Detection and Tracking Approach QuikSCAT Feature Selection (cont`d) 52 If λ1 and λ2 are the eigenvalues of matrix M such that λ1 < λ2, then the eigenvalue ratio of a bounding box B of dimension m by n is ERB is used to quantify the relative strength of the dominant wind direction (DOWD) within B 2/22/2010 Automated Cyclone Discovery and tracking Heterogeneous Remote Satellite-Based Detection and Tracking Approach QuikSCAT Feature Selection (cont`d) 53 If there is a circulation (i.e. a cyclone in B), ERB will be near to 1 If the wind is unidirectional (no storm or cyclone in B), λ2 will be much greater than λ1, and as a result ERB is much larger 2/22/2010 Automated Cyclone Discovery and tracking Heterogeneous Remote Satellite-Based Detection and Tracking Approach QuikSCAT Feature Selection (cont`d) The relative wind vorticity (RWV) at location (i,j) is calculated by the formula: where u and v are the two wind vector components in the west-east and south-north directions, and d is the spatial distance between two adjacent QuikSCAT measurements in a uniformly gridded data ωz or ζ: vertical component of relative vorticity 54 2/22/2010 Automated Cyclone Discovery and tracking Heterogeneous Remote Satellite-Based Detection and Tracking Approach Ensemble Classifier for Cyclone Detection 55 Ensemble methods are learning algorithms that make predictions on observations based on a majority or weighted vote from a set of classifiers or predictors 2/22/2010 Automated Cyclone Discovery and tracking Heterogeneous Remote Satellite-Based Detection and Tracking Approach Ensemble Classifier for Cyclone Detection (cont`d) The ensemble classifier is built to identify cyclones in QuikSCAT images The TRMM precipitation data are not used in the ensemble because – – 56 It has a weak discriminating power; heavy rainfall does not imply existence of cyclone It is very unlikely that one has QuikSCAT and TRMM data concurrently 2/22/2010 Automated Cyclone Discovery and tracking Heterogeneous Remote Satellite-Based Detection and Tracking Approach Ensemble Classifier for Cyclone Detection (cont`d) 57 Regions in a QuikSCAT image likely to contain a cyclone are localized based on wind speed Regions that have areas less than some threshold are removed Five classifiers based on features extracted from the QuikSCAT training data are constructed to identify the cyclones Two classifiers are thresholding classifier based on the DOWD and RWV features, and the other three are support vector machine (SVM) that use histogram features for WS, WD and DSR The classification decision is based on majority vote among the five classifiers 2/22/2010 Figure 5. Ensemble Classifier (Cyclone Discovery Module) Automated Cyclone Discovery and tracking Heterogeneous Remote Satellite-Based Detection and Tracking Approach Knowledge Sharing between TRMM and QuikSCAT data for Cyclone Tracking 58 The multi-sensor knowledge-sharing solution is based on the strength of each remote sensor type – QuikSCAT has excellent information for cyclone detection but lack sufficient temporal resolution (each pass-through is repeated only every 12 hours) – TRMM has excellent temporal resolution of 3 hours, but lacks good discriminative ability for accurate cyclone detection – Therefore, QuikSCAT data are used for cyclone detection, and TRMM data for tracking based on knowledge obtained from the ensemble classifier using QuikSCAT features 2/22/2010 Automated Cyclone Discovery and tracking Heterogeneous Remote Satellite-Based Detection and Tracking Approach Knowledge Sharing between TRMM and QuikSCAT data for Cyclone Tracking (cont`d) 59 QuikSCAT data are retrieved, and are input into the cylone discovery module to locate or identify possible cyclones The cyclone location is used to predict the likely regions to contain a cyclone at the next incoming data stream retrieved using a linear Kalman filter predictor, which is important because TRMM precipitation data are not a definitive indicator of cyclones A cyclone localized by applying a threshold to the TRMM precipitation rate measurement (T6 = 0) After a cyclone is located the Kalman filter measurement update or correction is applied to obtain an estimate of the new state vector or the predicted location of the cyclone in the next TRMM (or QuikSCAT) observation cycle 2/22/2010 Figure 6. Knowledge sharing between TRMM and QuikSCAT data for Cyclone Tracking Automated Cyclone Discovery and tracking Heterogeneous Remote Satellite-Based Detection and Tracking Approach Knowledge Sharing between TRMM and QuikSCAT data for Cyclone Tracking (cont`d) 60 A cyclone is a dynamic event and its size evolves rapidly over time, and therefore modeling and predicting only the cyclone center in space over time would be grossly inadequate Thus, the maximum and the minimum latitude/longitude of the bounding box spanned by the cyclone is used based on the hypothesis that the cyclone evolves linearly in space over time The estimated bounding box was expanded (or contracted) based on the estimated Kalman error covariance to define a search region for the cyclone in the TRMM image This modeling significantly improves the quality of knowledge sharing between heterogeneous satellites compare to the model that uses only the center coordinates of the cyclone 2/22/2010 Automated Cyclone Discovery and tracking Outline 61 Introduction Previous Work Data Description Issues and Challenges Heterogeneous Remote Satellite-Based Detection and Tracking Approach Experimental Results and Conclusions 2/23/2010 Automated Cyclone Discovery and tracking Experimental Results Training Set and Test Data Training Set – – Test Set – – 62 191 QuikSCAT images of cyclones occurring in North Atlantic Ocean in 2003 1833 negative examples (unlabeled examples from four days in 2003 that no tropical cyclone) 54 cyclone events in North Atlantic Ocean in 2006 1822 non-cyclone events 2/23/2010 Automated Cyclone Discovery and tracking Experimental Results Classification Performance 63 Step 1: Determine thresholds for DOWD (Dominant Wind Direction) and RWV (Dominant Wind Vorticity) features from test set results 2/23/2010 Automated Cyclone Discovery and tracking Experimental Results Performance of DOWD classifier 0.80 Positive 0.59 0.38 1.958 64 2/23/2010 Automated Cyclone Discovery and tracking Experimental Results Performance of RWV Classifier 0.89 0.85 0.80 1.51 65 2/23/2010 Automated Cyclone Discovery and tracking Experimental Results Classification Performance 66 Step 1: Determine thresholds for DOWD (Dominant Wind Direction) and RWV (Dominant Wind Vorticity) features from test set results Step 2: Analyze performance of different classifier ensembles 2/23/2010 Automated Cyclone Discovery and tracking Experimental Results Different Classifier Ensembles •RWV •DOWD •SVM ensemble •CDM •SVM + RWV ensemble •SVM + DOWD ensemble •CIS (Ho and Talukder, 2008) Experimental Results 68 2/23/2010 Automated Cyclone Discovery and tracking Experimental Results ROC Curve (Receiver Operating Characteristics) •RWV is a more robust feature than DOWD in discriminating cyclone and non-cyclone events 69 2/23/2010 Automated Cyclone Discovery and tracking Experimental Results Classifier Performance 70 Step 1: Determine thresholds for DOWD (Dominant Wind Direction) and RWV (Dominant Wind Vorticity) features from test set results Step 2: Analyze performance of classifiers Step 3: Use CDM to track an isolated hurricane event (Hurricane Isabel, 2003) using QuikSCAT and TRMM data 2/23/2010 Automated Cyclone Discovery and tracking Experimental Results Tracking Hurricane Isabel 71 2/23/2010 Automated Cyclone Discovery and tracking Experimental Results Tracking Hurricane Isabel 72 2/23/2010 Automated Cyclone Discovery and tracking Experimental Results Tracking Hurricane Isabel 73 2/23/2010 Automated Cyclone Discovery and tracking Experimental Results Tracking Hurricane Isabel 74 2/23/2010 Automated Cyclone Discovery and tracking Experimental Results Tracking Hurricane Isabel 75 2/23/2010 Automated Cyclone Discovery and tracking Experimental Results Experimental Results Tracking Hurricane Isabel 76 2/23/2010 Automated Cyclone Discovery and tracking Experimental Results Tracking Hurricane Isabel 77 2/23/2010 Automated Cyclone Discovery and tracking Experimental Results Tracking Hurricane Isabel 78 2/23/2010 Automated Cyclone Discovery and tracking Experimental Results Tracking Hurricane Isabel 79 2/23/2010 Automated Cyclone Discovery and tracking Experimental Results Tracking Hurricane Isabel 80 2/23/2010 Automated Cyclone Discovery and tracking Experimental Results Tracking Hurricane Isabel 81 2/23/2010 Automated Cyclone Discovery and tracking Experimental Results Tracking Hurricane Isabel 82 2/23/2010 Automated Cyclone Discovery and tracking Experimental Results Tracking Hurricane Isabel 83 2/23/2010 Automated Cyclone Discovery and tracking Experimental Results Tracking Hurricane Isabel 84 2/23/2010 Automated Cyclone Discovery and tracking Experimental Results Tracking Hurricane Isabel 85 2/23/2010 Automated Cyclone Discovery and tracking Experimental Results Tracking Hurricane Isabel 86 2/23/2010 Automated Cyclone Discovery and tracking Conclusions 87 Conventional methods that utilize human resources cannot handle massive, unlabeled high-dimensional heterogeneous data This method provides an efficient solution to track cyclonic events which combines information from multiple satellites The threshold values depend on the desired accuracy, as well as the desired rate of true positives and true negatives Automated Cyclone Discovery 2/23/2010 and tracking Questions? 88 2/23/2010 Automated Cyclone Discovery and tracking