Ecosystem Model-Data Intercomparison Workshop Outlier Detection and Flagging for Class B Sites R.J. Olson, J.M.O. Scurlock, K.J. Johnson, and EMDI Workshop Participants January 27, 2000 Summary Assigned consistent biome class to Class B sites (2363 records, 1271 sites) Calculated new NPP ensemble values (8 models) Calculated new AET ensemble values (4 models) Performed outlier analysis on NPP and driver data (18 flags) Excluded managed sites (crops, pasture, plantation and wetlands) (169 sites) Identified a set of 373of the 2194 (2363 –169) sites to be excluded, including: One or more critical flags: 189 flagged related to differences with NPP ensemble (bias, NE, MAE) 180 flagged related to high elevation (>2500 m) 49 flagged related to big differences between site and global precipitation 35 flagged related to big differences between site and global temperature Five or more less critical flags: Outside of .05 to .95 percentile by biome Outside of .95 CI determined by regressions with AET, PREC, TAVE Recalculated NPP for 959 sites from the 1821 NPP records Compared revised NPP with model ensemble NPP, modeled ensemble averaged 91 gC/m2 higher than observed NPP Background The Ecosystem Model-Data Intercomparison (EMDI) Workshop was held December 5-8, 1999 in Durham, New Hampshire with 12 modeling groups participating. The EMDI Workshop included a variety of models, including biogeochemical, satellite-driven, detailed process, and DVGM types. Initial results showed general agreement between models and data but with obvious differences that indicate areas for potential data and model improvement. Much of the workshop was devoted to looking at potential outliers and harmonizing some of the driver data, especially land cover or vegetation types. Goal - The goal is to produce a consistent set of NPP measurements with associated environmental driver data that can be used for regional model development and validation. QA Issues - The ORNL group had reviewed the data extensively prior to EMDI by looking at scatter plots and data outside of reasonable limits. However, the Workshop provided the initial model results to compare with the observed NPP data. An ensemble NPP value was calculated for each site as the average of the 12 Class A (old class 1-2) and 8 Class B (old Class 3) models, including AVIM, CARAIB, CENTURY, GLO-PEM, IBIS, PnET, STOMATE, and VECODE. In addition, an ensemble AET (actual evapotranspiration) value was calculated based on the average of AET provided by four of the models. The EMDI workshop also provided an opportunity to review the classification of the sites, compare observed to predicted NPP, and look at relationships between variables. The specific issues that we hope to address are: Land cover / Biome Class Consistency- This review was prompted in part by the realization of problems in using the satellite-derived land cover for each site (i.e., often this represented 1 the dominant land cover for a 1x1 km area, not the typical 1x1 m to 1 hectare NPP measurement site). Managed sites - In addition, modelers decided to flag and exclude likely heavily managed sites and wetlands from the EMDI comparison. Multiple NPP values for a site – Some sites have up to 35 observed NPP values, often from several vegetation types. We assume this is often a result of reporting imprecise latitude/longitude coordinates. Coordinates – Coordinates for sites are reported in a mix of formats, from whole degrees for a set of Russian sites to 4 decimal places for GPS registered sites. Some of the climate data and model outputs were provided with coordinates rounded off to 2 decimal places. Although we had distributed data with coordinates up to 4 decimal places, we rounded all sites to 2 decimal places for consistency. The power of the statistical-empirical approach is that we can look for patterns within similar groups (i.e., biomes) and look for relationships between variables. Even if we could review the original literature reference associated with each study, we would not pickup the potential outliers that were found in this process. Biomes - The outlier process started by reviewing the biome designation for all sites. Twentyone classes were defined at the EMDI workshop to represent the data and needs of the models. They were assigned based on initial biome class, subbiome, species, vegetation type and evaluation by Jonathan Scurlock, Peter Thornton, Mac Post, Bill Parton, Steve DeGrosa and others. Four classes were assigned to heavily managed or sites typically not addressed by regional models. The types included crops, pasture, plantations, and wetlands. These sites will be flagged and generally excluded from the EMDI exercises. The 21 biome classes were grouped into 12 classes based on the number of sites to ensure there were enough data (at least 30-40 sites) within each biome to conduct the outlier detection described below. Biomes: BIOMENEW *crops *pasture *plantation *wetland DBL forest / boreal DBL forest / temperate DBL forest / tropical desert DNL forest / boreal EBL forest / temperate EBL forest / tropical ENL forest / boreal ENL forest / temperate grassland / C3 grassland / C4 temperate grassland / C4 tropical mediterranean mixed forest Total 14 17 27 46 43 233 17 26 29 250 102 117 210 41 18 32 12 49 Aggregated (BIOME2) managed managed managed managed boreal DBL forest / temperate DBL forest / tropical desert boreal EBL forest / temperate EBL forest / tropical ENL forest / boreal ENL forest / temperate grassland grassland grassland Savanna mixed forest 2 Savanna / temperate Savanna / tropical tundra Grand Total 1 8 24 1317 Savanna Savanna tundra Classes At the EMDI Workshop, we decided to rename the groups to Class A (old Class 1 &2), Class B (old Class 3) and Class C (old Regional cells. This analysis covers Class B sites. A similar approach will be used with Class A and Class C sites. Actually, we may use the biome level information derived from Class B sites to compare to the sites in the other classes. Class B Sites The EMDI NPP and driver data were reviewed to flag potential outliers based on criteria to identify unrepresentative sites or potential errors. Each variable was reviewed independently and then in combination with other variables. At the EMDI Workshop, the QA checking was restricted to average NPP at 918 Class B (old Class 3) sites, those for which a complete set of model predictions from 7 models were available. In this phase, we reviewed all 2363 individual NPP data values that had model predictions from at least 3 models. The final step will be to calculate average NPP for all unique site-biome combinations using the subset of NPP records that pass our flagging criteria. Initially there were 1663 Class B sites; however, this set was reduced to 1271 unique sites with valid driver data. We expect after the exclusion of outliers that there will be approximately 1000 sites remaining. The approach included the following tests to set flags: 1. Flags based on values outside of reasonable limits: ANPP_C>2000 gC/m2, BNPP_C>2000, TNPP_C > 3000 ELEV > 2500 m (high elevation sites were expected to comprise unrepresentative subbiomes or present problems with the climate extrapolation algorithm) ANPP > .95TNPP, BNPP>.95 TNPP (for a site with both above and below ground components) 2. Flags based on questionable values. <We didn’t include this in our set of flags but are aware that the Russian sites should be flagged and there may be other sites.> LAT_DD, LONG_DD 3. Flags based on NPP values outside of the .05-.95 percentiles for each biome calculated assuming a normal distribution of variables (this replaces our initial rule of using 2 standard deviations): ANPP, BNPP, TNPP 4. Flags based on climate values outside of the .01-.99 percentiles for each biome calculated assuming a normal distribution of variables. In addition, we set some specific limits for some biomes: 3 Temp, Precip Temp > 6 C for boreal forests Precip > 1000 mm for desert and tundra Precip < 1000 mm for tropical forest 5. Flags based on inconsistencies at a site, such as precipitation reported for the site being different than the precipitation derived from global climate data. These flags were based on calculating the Normalized Error (NE) as the ratio of the difference (predicted - observed) divided by the average (predicted + observed )/2. Based on the frequency distribution of the ratios, ratios greater than 1.0 were flagged. Elevation, Precipitation, Temperature 6. Flags based on the comparison of measured NPP versus modeled NPP using the average or ensemble value for all available models. The comparison was based on bias (predicted observed), Normalized Error (predicted - observed) divided by the average (predicted observed )/2, and Mean Absolute Error (MAE) – (predicted - observed) divided by the observed. . Based on the frequency distributions, bias greater than +_ 1000 gC/m2, NE ratios greater than +_1.0, and MAE ratios greater than +_ 5 were flagged. NPP vs MODEL ensemble 7. Flags based on relationships between variables. Linear regression analysis was performed between NPP and average AET (from 4 models), NPP and precipitation, and NPP and temperature. Points falling outside of the .95 Confidence Interval about the regression line were flagged. NPP vs AET, NPP vs Prec, NPP vs Tave Critical flags We assume that sites that have multiple flags have inconsistencies between NPP, driver data, and model predictions, and are therefore more likely to be an outlier. This seemed to hold true in plots (see below). We also know that some flags are deserve more critical weight as indicators of potential problems. We designated flags by assigning a value of 10 for those critical checks, including NPP inconsistent with model NPP ensemble value, elevation > 2500 m or < -100 m, and either site temperature or precipitation inconsistent with that assigned based on the global climate data. We assigned a flag of 100 to the heavily managed sites for easy identification. The overall flags value was calculated as the sum of the individual flags. Sites with a sum greater that 100 (managed biomes) were dropped, and those greater than 5 (all those with at least one major flag) were considered potential for excluding. 1. Drop managed sites (169 or 7% of the total) 2. Select sites with no flags set or less than 5 noncritical flags (or exclude those with a critical flag set or five or more noncritical flags) Results 4 A total of 136 of the 2363 records were dropped because they are managed sites. A total of 373 appeared to be outliers. We are in the process of reviewing this list. Virtually all the sites identified at the EMDI workshop as outliers were included in this set. If the EMDI group as a whole accepted these as outliers and we recalculate site and biome averages, we would have a total of 1821 records for 951 sites. Estimating Total NPP For those sites lacking a value for TNPP, we calculated biome-specific ratios to estimate NPP_EST from ANPP or BNPP. We will recalculate these ratios based on the new biome classification and recalculate NPP_EST use data with outliers excluded. Plots and Charts We plotted NPP observed against the modeled NPP ensemble, by AET ensemble, and by latitude with the plot symbols indicating the magnitude of the flags. In general, the points that had high flag values appeared to be on the fringe of the cluster of points. As included below, the difference between the old NPP and revised included increases and decreases, generally getting bigger with higher NPP levels. There was a strong relationship of the recalculated model ensemble to be higher at low NPP and lower than observed NPP at higher NPP. Bar charts are presented for the new biome groups. The SAS variable names and labels for the flags are: Flag name BIOME_F ELEV_MXF PREC_F TAVE_F MOD_F ANPP_P5F BNPP_P5F TNPP_P5F NPP_P5F ANPPBADF BNPPBADF TNPPBADF AET_RGF PREC_RGF TAVE_RGF ELEV_F PRECBADF TAVEBADF Flag 100 10 10 10 10 1 1 1 1 1 1 1 1 1 1 1 1 1 Flag label crops, pasture, plantations, wetlands** elev >2500, elev < -100* (prec – prec_ann) / ave(prec) > 1* (tave – temp_ann) / ave(tave) > 1* (npp_est - modcb_av) / ave(npp) > 1* anpp outsite .05-.95 percentile by biome2 bnpp outsite .05-.95 percentile by biome2 tnpp outsite .05-.95 percentile by biome2 npp outsite .05-.95 percentile by biome2 anpp >2000gc/m2, anpp>95%tnpp bnpp >2000gc/m2, bnpp>95%tnpp tnpp >3000gc/m2, anppbadf or bnppbadf NPP outside .95 CI of NPP=a+b*AET regression NPP outside .95 CI of NPP=a+b*PREC regression NPP outside .95 CI of NPP=a+b*TAVE regression (elev_giv - elev_dem) / ave(elev) > 1 prec outsite .01-.99 percentile by biome2 (or >1000mm in desert) temp outsite .01-.99 percentile by biome2 (>6 in boreal forests) 5 EMDI Outlier Analysis January 28, 2000 202 ---------------------------------------------------------------------------| | N |NMISS | MEAN | STD |STDERR| MIN | MAX | |-------------------------+------+------+------+------+------+------+------|NPP_EST |BIOMENEW | | | | | | | | | |------------| | | | | | | | |*crops | 26| 0| 1137| 673| 132| 210| 3100| | |------------+------+------+------+------+------+------+------| |*pasture | 25| 0| 1327| 814| 163| 120| 3000| | |------------+------+------+------+------+------+------+------| |*plantation | 35| 0| 888| 410| 69| 200| 1860| | |------------+------+------+------+------+------+------+------| |*wetland | 83| 0| 638| 597| 66| 60| 2920| | |------------+------+------+------+------+------+------+------| |DBL forest /| | | | | | | | | |boreal | 48| 0| 596| 259| 37| 100| 1140| | |------------+------+------+------+------+------+------+------| |DBL forest /| | | | | | | | | |temperate | 392| 0| 592| 251| 13| 110| 2187| | |------------+------+------+------+------+------+------+------| |DBL forest /| | | | | | | | | |tropical | 46| 0| 653| 365| 54| 140| 1520| | |------------+------+------+------+------+------+------+------| |DNL forest /| | | | | | | | | |boreal | 43| 0| 509| 253| 39| 150| 1440| | |------------+------+------+------+------+------+------+------| |EBL forest /| | | | | | | | | |temperate | 375| 0| 796| 336| 17| 53| 2360| | |------------+------+------+------+------+------+------+------| |EBL forest /| | | | | | | | | |tropical | 168| 0| 922| 380| 29| 60| 2093| ---------------------------------------------------------------------------Œ (CONTINUED) 7 EMDI Outlier Analysis January 28, 2000 203 ---------------------------------------------------------------------------| | N |NMISS | MEAN | STD |STDERR| MIN | MAX | |-------------------------+------+------+------+------+------+------+------|NPP_EST |BIOMENEW | | | | | | | | | |------------| | | | | | | | |ENL forest /| | | | | | | | | |boreal | 225| 0| 356| 213| 14| 10| 1120| | |------------+------+------+------+------+------+------+------| |ENL forest /| | | | | | | | | |temperate | 447| 0| 722| 468| 22| 53| 2413| | |------------+------+------+------+------+------+------+------| |Savanna / | | | | | | | | | |temperate | 1| 0| 450| .| .| 450| 450| | |------------+------+------+------+------+------+------+------| |Savanna / | | | | | | | | | |tropical | 8| 0| 434| 434| 153| 60| 1390| | |------------+------+------+------+------+------+------+------| |desert | 43| 0| 141| 191| 29| 20| 810| | |------------+------+------+------+------+------+------+------| |grassland / | | | | | | | | | |C3 | 68| 0| 332| 237| 29| 40| 1110| | |------------+------+------+------+------+------+------+------| |grassland / | | | | | | | | | |C4 temperate| 32| 0| 388| 219| 39| 30| 1080| | |------------+------+------+------+------+------+------+------| |grassland / | | | | | | | | | |C4 tropical | 61| 0| 687| 541| 69| 40| 2260| | |------------+------+------+------+------+------+------+------| |mediterrane-| | | | | | | | | |an | 12| 0| 422| 259| 75| 80| 800| ---------------------------------------------------------------------------Œ (CONTINUED) 8 EMDI Outlier Analysis January 28, 2000 204 ---------------------------------------------------------------------------| | N |NMISS | MEAN | STD |STDERR| MIN | MAX | |-------------------------+------+------+------+------+------+------+------|NPP_EST |BIOMENEW | | | | | | | | | |------------| | | | | | | | |mixed forest| 173| 0| 550| 278| 21| 80| 2160| | |------------+------+------+------+------+------+------+------| |tundra | 52| 0| 101| 82| 11| 10| 370| | |------------+------+------+------+------+------+------+------| |ALL | 2363| 0| 638| 419| 9| 10| 3100| ---------------------------------------------------------------------------Œ 9 EMDI Outlier Analysis January 28, 2000 205 ---------------------------------------------------------------------------| | N |NMISS | MEAN | STD |STDERR| MIN | MAX | |-------------------------+------+------+------+------+------+------+------|ANPP_C |ALL | 1484| 879| 429| 323| 8| 10| 2180| |------------+------------+------+------+------+------+------+------+------|BNPP_C | | 601| 1762| 151| 175| 7| 10| 2770| |------------+------------+------+------+------+------+------+------+------|TNPP_C | | 1486| 877| 638| 373| 10| 10| 3100| |------------+------------+------+------+------+------+------+------+------|NPP_EST | | 2363| 0| 638| 419| 9| 10| 3100| |------------+------------+------+------+------+------+------+------+------|PREC | | 2363| 0| 1041| 579| 12| 36| 3778| |------------+------------+------+------+------+------+------+------+------|TAVE | | 2363| 0| 10| 8| 0| -18| 29| |------------+------------+------+------+------+------+------+------+------|ELEV | | 2363| 0| 805| 882| 18| -36| 4240| ---------------------------------------------------------------------------Œ 10 EMDI Outlier Analysis B I O M E 2 DBL forest / DBL forest / EBL forest / EBL forest / ENL forest / ENL forest / boreal desert grassland managed mixed forest savanna tundra All tempera tropica tempera tropica boreal tempera January 28, 2000 206 A N P P _ N A N P P _ C A N P P _ P 5 237 40 44 79 206 345 63 32 148 88 144 18 40 1484 479 512 718 662 205 602 297 53 241 508 431 183 74 429 190 150 320 250 30 130 90 10 40 80 100 30 15 50 A N P P _ P 9 5 T N P P _ N 900 1220 1700 1370 450 1360 480 160 730 1110 830 400 205 1030 255 35 340 127 101 265 41 22 103 98 69 5 25 1486 T N P P _ C T N P P _ P 5 T N P P _ P 9 5 598 714 775 890 312 602 492 201 513 844 540 644 87 638 290 190 380 400 100 200 180 30 100 170 260 340 10 140 1040 1510 1330 1440 590 1180 920 800 1370 2010 760 1390 200 1330 N P P _ N N P P _ E S T N P P _ P 5 N P P _ P 9 5 392 46 375 168 225 447 91 43 161 169 173 21 52 2363 592 653 796 921 356 722 555 141 478 869 550 428 101 638 260 190 380 410 60 190 180 20 80 140 173 80 15 100 1067 1350 1430 1500 820 1720 960 680 1370 2070 1080 800 270 1440 11 EMDI Outlier Analysis B I O M E 2 DBL forest / DBL forest / EBL forest / EBL forest / ENL forest / ENL forest / boreal desert grassland managed mixed forest savanna tundra All tempera tropica tempera tropica boreal tempera January 28, 2000 207 E L E V _ N E L E V E L E V _ P 1 392 46 375 168 225 447 91 43 161 169 173 21 52 2363 604 149 1386 567 490 1090 630 1111 624 467 627 418 872 805 10 79 50 53 60 6 92 68 28 2 -1 53 -2 5 E L E V _ P 9 9 2805 1290 3800 2163 1530 3900 1946 3362 2367 2750 2900 1502 3042 3642 P R E C P R E C _ N P R E C _ P 1 P R E C _ P 9 9 T A V E 935 1228 1288 1813 581 1150 616 330 823 1093 985 620 652 1041 392 46 375 168 225 447 91 43 161 169 173 21 52 2363 104 654 593 943 252 199 175 115 167 162 136 256 41 105 2535 1565 2550 3778 1262 3096 1637 789 2315 3464 1890 1277 1748 3096 7.92 25.91 12.51 21.06 0.44 9.33 0.03 13.63 15.06 14.02 10.61 17.05 -6.29 10.18 T A V E _ P 1 T A V E _ P 9 9 392.00 -5.00 46.00 20.00 375.00 2.00 168.00 13.00 225.00 -7.00 447.00 -5.00 91.00 -10.00 43.00 -1.00 161.00 -3.00 169.00 -2.00 173.00 0.00 21.00 6.00 52.00 -18.00 2363.0 -9.00 26.00 27.00 18.00 28.00 5.00 24.00 9.00 26.00 28.00 27.00 19.00 27.00 6.00 27.00 T A V E _ N 12 EMDI Outlier Analysis January 28, 2000 208 -----------------------------------------------| | N | SUM |NMISS | |-------------------------+------+------+------|**biome=cro-|ALL | | | | |ps, pasture,| | | | | |wetlands, | | | | | |etc | | 2363| 16900| 0| |------------+------------+------+------+------|*(npp_est - | | | | | |modcb_av) / | | | | | |ave(npp) > 1| | 2326| 1890| 37| |------------+------------+------+------+------|*(prec | | | | | |prec_ann) / | | | | | |ave(prec) > | | | | | |1 | | 787| 490| 1576| |------------+------------+------+------+------|*(tave | | | | | |temp_ann) / | | | | | |ave(tave) > | | | | | |1 | | 607| 350| 1756| |------------+------------+------+------+------|*elev >2500 | | | | | |or elev < - | | | | | |100 | | 2363| 1800| 0| |------------+------------+------+------+------|(elev_giv - | | | | | |elev_dem) / | | | | | |ave(elev) > | | | | | |1 | | 2363| 47| 0| -----------------------------------------------Œ (CONTINUED) 13 EMDI Outlier Analysis January 28, 2000 209 -----------------------------------------------| | N | SUM |NMISS | |-------------------------+------+------+------|anpp outsite|ALL | | | | |.05 | | | | | |percentile | | | | | |by biome2 | | 1484| 127| 879| |------------+------------+------+------+------|bnpp outsite| | | | | |.05 | | | | | |percentile | | | | | |by biome2 | | 601| 41| 1762| |------------+------------+------+------+------|tnpp outsite| | | | | |.05 | | | | | |percentile | | | | | |by biome2 | | 1486| 132| 877| |------------+------------+------+------+------|npp outsite | | | | | |.05 | | | | | |percentile | | | | | |by biome2 | | 2363| 215| 0| |------------+------------+------+------+------|anpp>2000gc-| | | | | |/m2 or | | | | | |anpp>95%tnpp| | | | | |or < 50 | | 1484| 51| 879| -----------------------------------------------Œ (CONTINUED) 14 EMDI Outlier Analysis January 28, 2000 210 -----------------------------------------------| | N | SUM |NMISS | |-------------------------+------+------+------|bnpp>2000gc-|ALL | | | | |/m2 or | | | | | |bnpp>95%tnpp| | | | | |or < 50 | | 601| 90| 1762| |------------+------------+------+------+------|tnpp>3000gc-| | | | | |/m2 or | | | | | |anppbadf or | | | | | |bnppbadf | | 1486| 102| 877| |------------+------------+------+------+------|prec | | | | | |unreasonable| | | | | |for biome2 | | 2363| 237| 0| |------------+------------+------+------+------|temp | | | | | |unreasonable| | | | | |for biome2 | | 2363| 78| 0| |------------+------------+------+------+------|npp value | | | | | |outside | | | | | |npp=a+b*AET | | | | | |.95CI | | 1635| 37| 728| |------------+------------+------+------+------|npp value | | | | | |outside | | | | | |npp=a+b*PREC| | | | | |.95CI | | 1649| 39| 714| -----------------------------------------------Œ (CONTINUED) 15 EMDI Outlier Analysis January 28, 2000 211 -----------------------------------------------| | N | SUM |NMISS | |-------------------------+------+------+------|npp value |ALL | | | | |outside | | | | | |npp=a+b*TAVE| | | | | |.95CI | | 1649| 41| 714| -----------------------------------------------Œ 16 EMDI Outlier Analysis B I O M E 2 DBL forest / DBL forest / EBL forest / EBL forest / ENL forest / ENL forest / boreal desert grassland mixed forest savanna tundra All tempera tropica tempera tropica boreal tempera January 28, 2000 212 A N P P _ N A N P P _ C A N P P _ P 5 230 38 34 67 172 289 59 16 124 116 14 23 1182 472 464 603 681 213 578 290 84 222 440 221 84 421 210 150 320 290 70 220 90 30 50 180 40 20 80 A N P P _ P 9 5 T N P P _ N 860 1020 950 1180 420 1130 480 390 610 760 400 180 950 242 32 261 118 90 179 39 14 87 56 5 14 1137 T N P P _ C T N P P _ P 5 T N P P _ P 9 5 592 682 837 923 326 653 505 217 486 512 644 94 648 290 190 430 420 140 300 180 30 120 310 340 20 180 1010 1270 1430 1470 590 1090 1000 800 1090 750 1390 200 1270 N P P _ N N P P _ E S T N P P _ P 5 N P P _ P 9 5 375 43 290 154 191 336 87 21 134 145 17 28 1821 588 625 836 945 368 724 545 193 449 551 505 121 633 270 190 427 430 140 290 200 60 110 253 80 30 180 1040 1260 1430 1480 720 1470 940 680 1090 907 1390 270 1270 17 EMDI Outlier Analysis B I O M E 2 DBL forest / DBL forest / EBL forest / EBL forest / ENL forest / ENL forest / boreal desert grassland mixed forest savanna tundra All tempera tropica tempera tropica boreal tempera January 28, 2000 213 E L E V _ N E L E V E L E V _ P 1 375 43 290 154 191 336 87 21 134 145 17 28 1821 529 153 923 552 516 656 641 1018 585 505 339 560 619 10 79 50 53 60 6 92 68 30 -1 53 -2 10 E L E V _ P 9 9 2188 1290 2414 2163 1625 2310 1946 2464 2367 1970 1095 2218 2300 P R E C P R E C _ N P R E C _ P 1 948 1227 1354 1805 583 1243 615 246 832 1014 634 370 1069 375 43 290 154 191 336 87 21 134 145 17 28 1821 107 654 737 943 252 223 175 115 186 320 301 41 105 P R E C _ P 9 9 T A V E _ P 1 T A V E _ P 9 9 2535 7.76 375.00 -5.00 1565 25.88 43.00 20.00 2138 13.73 290.00 6.00 3778 21.03 154.00 13.00 1262 0.61 191.00 -7.00 3096 10.57 336.00 -1.00 1637 0.03 87.00 -10.00 647 13.52 21.00 6.00 2315 14.61 134.00 -3.00 1890 10.74 145.00 0.00 1277 17.18 17.00 6.00 1748 -10.18 28.00 -18.00 3096 10.28 1821.0 -12.00 25.00 27.00 18.00 28.00 5.00 25.00 9.00 26.00 28.00 19.00 27.00 6.00 27.00 T A V E T A V E _ N 18 EMDI Outlier Analysis January 28, 2000 214 ---------------------------------------------------------------------------| | N |NMISS | MEAN | STD |STDERR| MIN | MAX | |-------------------------+------+------+------+------+------+------+------|NPP_EST |BIOMENEW | | | | | | | | | |------------| | | | | | | | |DBL forest /| | | | | | | | | |boreal | 46| 0| 591| 249| 37| 100| 960| | |------------+------+------+------+------+------+------+------| |DBL forest /| | | | | | | | | |temperate | 375| 0| 588| 233| 12| 110| 1547| | |------------+------+------+------+------+------+------+------| |DBL forest /| | | | | | | | | |tropical | 43| 0| 625| 317| 48| 187| 1350| | |------------+------+------+------+------+------+------+------| |DNL forest /| | | | | | | | | |boreal | 41| 0| 495| 205| 32| 180| 1130| | |------------+------+------+------+------+------+------+------| |EBL forest /| | | | | | | | | |temperate | 290| 0| 836| 292| 17| 200| 1670| | |------------+------+------+------+------+------+------+------| |EBL forest /| | | | | | | | | |tropical | 154| 0| 945| 342| 28| 330| 1910| | |------------+------+------+------+------+------+------+------| |ENL forest /| | | | | | | | | |boreal | 191| 0| 368| 169| 12| 80| 900| | |------------+------+------+------+------+------+------+------| |ENL forest /| | | | | | | | | |temperate | 336| 0| 724| 348| 19| 67| 1813| | |------------+------+------+------+------+------+------+------| |Savanna / | | | | | | | | | |temperate | 1| 0| 450| .| .| 450| 450| ---------------------------------------------------------------------------Œ (CONTINUED) 19 EMDI Outlier Analysis January 28, 2000 215 ---------------------------------------------------------------------------| | N |NMISS | MEAN | STD |STDERR| MIN | MAX | |-------------------------+------+------+------+------+------+------+------|NPP_EST |BIOMENEW | | | | | | | | | |------------| | | | | | | | |Savanna / | | | | | | | | | |tropical | 6| 0| 552| 444| 181| 180| 1390| | |------------+------+------+------+------+------+------+------| |desert | 21| 0| 193| 198| 43| 30| 800| | |------------+------+------+------+------+------+------+------| |grassland / | | | | | | | | | |C3 | 59| 0| 356| 242| 32| 80| 1110| | |------------+------+------+------+------+------+------+------| |grassland / | | | | | | | | | |C4 temperate| 30| 0| 399| 216| 39| 90| 1080| | |------------+------+------+------+------+------+------+------| |grassland / | | | | | | | | | |C4 tropical | 45| 0| 604| 404| 60| 40| 1400| | |------------+------+------+------+------+------+------+------| |mediterrane-| | | | | | | | | |an | 10| 0| 482| 240| 76| 80| 800| | |------------+------+------+------+------+------+------+------| |mixed forest| 145| 0| 551| 215| 18| 173| 1520| | |------------+------+------+------+------+------+------+------| |tundra | 28| 0| 121| 82| 16| 20| 345| | |------------+------+------+------+------+------+------+------| |ALL | 1821| 0| 633| 333| 8| 20| 1910| ---------------------------------------------------------------------------Œ 20 EMDI Outlier Analysis January 28, 2000 216 ---------------------------------------------------------------------------| | N |NMISS | MEAN | STD |STDERR| MIN | MAX | |-------------------------+------+------+------+------+------+------+------|ANPP_C |ALL | 1182| 639| 421| 264| 8| 20| 1430| |------------+------------+------+------+------+------+------+------+------|BNPP_C | | 470| 1351| 141| 117| 5| 10| 790| |------------+------------+------+------+------+------+------+------+------|TNPP_C | | 1137| 684| 648| 320| 9| 20| 1910| |------------+------------+------+------+------+------+------+------+------|NPP_EST | | 1821| 0| 633| 333| 8| 20| 1910| |------------+------------+------+------+------+------+------+------+------|PREC | | 1821| 0| 1069| 584| 14| 36| 3778| |------------+------------+------+------+------+------+------+------+------|TAVE | | 1821| 0| 10| 8| 0| -18| 28| |------------+------------+------+------+------+------+------+------+------|ELEV | | 1821| 0| 619| 559| 13| -36| 2500| ---------------------------------------------------------------------------Œ 21 EMDI Outlier Analysis January 28, 2000 217 --------------------------------------------------------------------| | N |NMISS | MEAN | STD | MIN | MAX | |-------------------------+------+------+------+------+------+------|NPP_EST |BIOMENEW | | | | | | | | |------------| | | | | | | |DBL forest /| | | | | | | | |boreal | 29| 0| 582| 248| 100| 960| | |------------+------+------+------+------+------+------| |DBL forest /| | | | | | | | |temperate | 199| 0| 592| 215| 110| 1353| | |------------+------+------+------+------+------+------| |DBL forest /| | | | | | | | |tropical | 12| 0| 564| 212| 187| 830| | |------------+------+------+------+------+------+------| |DNL forest /| | | | | | | | |boreal | 28| 0| 498| 158| 210| 815| | |------------+------+------+------+------+------+------| |EBL forest /| | | | | | | | |temperate | 215| 0| 853| 281| 380| 1670| | |------------+------+------+------+------+------+------| |EBL forest /| | | | | | | | |tropical | 98| 0| 932| 314| 330| 1910| | |------------+------+------+------+------+------+------| |ENL forest /| | | | | | | | |boreal | 78| 0| 348| 169| 97| 900| | |------------+------+------+------+------+------+------| |ENL forest /| | | | | | | | |temperate | 139| 0| 632| 298| 67| 1676| | |------------+------+------+------+------+------+------| |Savanna / | | | | | | | | |temperate | 1| 0| 450| .| 450| 450| --------------------------------------------------------------------Œ (CONTINUED) 22 EMDI Outlier Analysis January 28, 2000 218 --------------------------------------------------------------------| | N |NMISS | MEAN | STD | MIN | MAX | |-------------------------+------+------+------+------+------+------|NPP_EST |BIOMENEW | | | | | | | | |------------| | | | | | | |Savanna / | | | | | | | | |tropical | 6| 0| 552| 444| 180| 1390| | |------------+------+------+------+------+------+------| |desert | 17| 0| 192| 191| 45| 800| | |------------+------+------+------+------+------+------| |grassland / | | | | | | | | |C3 | 36| 0| 358| 211| 95| 795| | |------------+------+------+------+------+------+------| |grassland / | | | | | | | | |C4 temperate| 16| 0| 454| 207| 120| 1080| | |------------+------+------+------+------+------+------| |grassland / | | | | | | | | |C4 tropical | 24| 0| 525| 387| 40| 1370| | |------------+------+------+------+------+------+------| |mediterrane-| | | | | | | | |an | 9| 0| 467| 249| 80| 800| | |------------+------+------+------+------+------+------| |mixed forest| 38| 0| 536| 167| 244| 1107| | |------------+------+------+------+------+------+------| |tundra | 14| 0| 124| 76| 30| 270| | |------------+------+------+------+------+------+------| |ALL | 959| 0| 637| 322| 30| 1910| --------------------------------------------------------------------Œ (CONTINUED) 23 EMDI Outlier Analysis January 28, 2000 219 --------------------------------------------------------------------| | N |NMISS | MEAN | STD | MIN | MAX | |-------------------------+------+------+------+------+------+------|MODCB_AV |BIOMENEW | | | | | | | | |------------| | | | | | | |DBL forest /| | | | | | | | |boreal | 26| 3| 648| 149| 175| 782| | |------------+------+------+------+------+------+------| |DBL forest /| | | | | | | | |temperate | 196| 3| 705| 155| 133| 1278| | |------------+------+------+------+------+------+------| |DBL forest /| | | | | | | | |tropical | 12| 0| 731| 124| 430| 892| | |------------+------+------+------+------+------+------| |DNL forest /| | | | | | | | |boreal | 28| 0| 503| 160| 215| 856| | |------------+------+------+------+------+------+------| |EBL forest /| | | | | | | | |temperate | 215| 0| 899| 92| 539| 1160| | |------------+------+------+------+------+------+------| |EBL forest /| | | | | | | | |tropical | 98| 0| 1026| 144| 625| 1415| | |------------+------+------+------+------+------+------| |ENL forest /| | | | | | | | |boreal | 78| 0| 560| 121| 303| 809| | |------------+------+------+------+------+------+------| |ENL forest /| | | | | | | | |temperate | 139| 0| 697| 225| 126| 1234| | |------------+------+------+------+------+------+------| |Savanna / | | | | | | | | |temperate | 1| 0| 720| .| 720| 720| --------------------------------------------------------------------Œ (CONTINUED) 24 EMDI Outlier Analysis January 28, 2000 220 --------------------------------------------------------------------| | N |NMISS | MEAN | STD | MIN | MAX | |-------------------------+------+------+------+------+------+------|MODCB_AV |BIOMENEW | | | | | | | | |------------| | | | | | | |Savanna / | | | | | | | | |tropical | 6| 0| 460| 162| 282| 746| | |------------+------+------+------+------+------+------| |desert | 17| 0| 222| 126| 72| 539| | |------------+------+------+------+------+------+------| |grassland / | | | | | | | | |C3 | 36| 0| 490| 173| 159| 839| | |------------+------+------+------+------+------+------| |grassland / | | | | | | | | |C4 temperate| 16| 0| 639| 235| 152| 927| | |------------+------+------+------+------+------+------| |grassland / | | | | | | | | |C4 tropical | 22| 2| 676| 237| 72| 1143| | |------------+------+------+------+------+------+------| |mediterrane-| | | | | | | | |an | 9| 0| 519| 175| 329| 787| | |------------+------+------+------+------+------+------| |mixed forest| 37| 1| 757| 158| 464| 1051| | |------------+------+------+------+------+------+------| |tundra | 9| 5| 304| 167| 93| 616| | |------------+------+------+------+------+------+------| |ALL | 945| 14| 738| 232| 72| 1415| --------------------------------------------------------------------Œ 25 EMDI Outlier Analysis January 28, 2000 221 ---------------------------------------------------------------------------| | N |NMISS | MEAN | STD |STDERR| MIN | MAX | |-------------------------+------+------+------+------+------+------+------|ANPP_C |ALL | 481| 478| 382| 241| 11| 20| 1353| |------------+------------+------+------+------+------+------+------+------|BNPP_C | | 200| 759| 146| 110| 8| 20| 570| |------------+------------+------+------+------+------+------+------+------|TNPP_C | | 688| 271| 671| 319| 12| 30| 1910| |------------+------------+------+------+------+------+------+------+------|NPP_EST | | 959| 0| 637| 322| 10| 30| 1910| |------------+------------+------+------+------+------+------+------+------|PREC | | 959| 0| 1031| 556| 18| 36| 3778| |------------+------------+------+------+------+------+------+------+------|TAVE | | 959| 0| 10| 8| 0| -18| 28| |------------+------------+------+------+------+------+------+------+------|ELEV | | 959| 0| 674| 563| 18| -36| 2500| ---------------------------------------------------------------------------Œ 26 EMDI Outlier Analysis January 28, 2000 222 --------------------------------------------------------------------| |Obser-| | | | | ved | | | | | NPP, | | | | |outli-| | | | | ers | | | | |dropp-|MODCB-| Predicted(ensemble) | | | ed | _AV | Observed NPP | | |------+------+---------------------------| | MEAN | MEAN | N | MEAN | STD | MAX | |-------------------------+------+------+------+------+------+------|BIOMENEW | | | | | | | |-------------------------| | | | | | |DBL forest / boreal | 415| 470| 9| 20| 259| 385| |-------------------------+------+------+------+------+------+------|DBL forest / temperate | 592| 705| 196| 109| 220| 569| |-------------------------+------+------+------+------+------+------|DBL forest / tropical | 598| 732| 11| 134| 200| 367| |-------------------------+------+------+------+------+------+------|DNL forest / boreal | 498| 503| 28| 5| 197| 235| |-------------------------+------+------+------+------+------+------|EBL forest / temperate | 853| 899| 215| 46| 265| 538| |-------------------------+------+------+------+------+------+------|EBL forest / tropical | 932| 1026| 98| 94| 330| 708| |-------------------------+------+------+------+------+------+------|ENL forest / boreal | 352| 528| 65| 177| 192| 546| |-------------------------+------+------+------+------+------+------|ENL forest / temperate | 632| 697| 139| 66| 268| 719| |-------------------------+------+------+------+------+------+------|Savanna / temperate | 450| 720| 1| 270| .| 270| --------------------------------------------------------------------Œ (CONTINUED) 27 EMDI Outlier Analysis January 28, 2000 223 --------------------------------------------------------------------| |Obser-| | | | | ved | | | | | NPP, | | | | |outli-| | | | | ers | | | | |dropp-|MODCB-| Predicted(ensemble) | | | ed | _AV | Observed NPP | | |------+------+---------------------------| | MEAN | MEAN | N | MEAN | STD | MAX | |-------------------------+------+------+------+------+------+------|BIOMENEW | | | | | | | |-------------------------| | | | | | |Savanna / tropical | 552| 460| 6| -91| 350| 207| |-------------------------+------+------+------+------+------+------|desert | 192| 222| 17| 30| 205| 409| |-------------------------+------+------+------+------+------+------|grassland / C3 | 358| 490| 36| 132| 172| 433| |-------------------------+------+------+------+------+------+------|grassland / C4 temperate | 454| 639| 16| 185| 179| 485| |-------------------------+------+------+------+------+------+------|grassland / C4 tropical | 525| 676| 22| 110| 366| 643| |-------------------------+------+------+------+------+------+------|mediterranean | 467| 519| 9| 52| 203| 287| |-------------------------+------+------+------+------+------+------|mixed forest | 536| 757| 37| 215| 221| 708| |-------------------------+------+------+------+------+------+------|tundra | 124| 304| 9| 160| 117| 346| |-------------------------+------+------+------+------+------+------|ALL | 641| 739| 914| 91| 255| 719| --------------------------------------------------------------------Œ 28 EMDI Outlier Analysis January 28, 2000 224 Plot of NPP_DIFF*NPP_EST. D i f f e r e n c e : O l d N e w N P P Legend: A = 1 obs, B = 2 obs, etc. 600 ˆ ‚ A A ‚ A ‚ A A ‚ A 400 ˆ ‚ ‚ AA A A ‚ A B ‚ B A A 200 ˆ A B A B A A A ‚ AA A A ‚ B A ‚ A A A A A ‚ B A A A A A A A A 0 ˆ A A A A AB AA A A B ‚ A A BCBA A A AABDD BDEA CCCD CBBBADC DABB C BAA A ‚ BCB A A AAA A A A A A AA A C A A A A ‚ A A CABB A ACA A ‚ A A B A -200 ˆ A AB ‚ A A B BAA A A ‚ A ‚ A ‚ A -400 ˆ Šƒˆƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒƒƒˆƒ 0 200 400 600 800 1000 1200 1400 1600 1800 Observed NPP, outliers dropped 29 EMDI Outlier Analysis January 28, 2000 225 Plot of NPP_MDIF*NPP_EST. Legend: A = 1 obs, B = 2 obs, etc. NPP_MDIF ‚ 1000 ˆ ‚ ‚ ‚ A ‚ A A B A A ‚ A AA A BA BBA A 500 ˆ BAB AB A BDDCB E D AA AA ‚ C A B D A BBB BAEABAAA AA A AA A ‚ AAEABABDCECCEBBFBFDDCDADCCC A A A ‚ BBB B GEBDBDADDGIFCDIDGEDDDBBBCBACDAA A A ‚ A AAABCCBD ABDEDFEBHADA DGFLCHHEDEDEEDDA A A A A ‚ ABA D A BBB CACCFAAAF AEGDDBFHCCCBDFA ACCAD AAAA A 0 ˆ AB BAAAB A ABBBEACACBDCEBGCDECCC BCBCCABCBD BC BAA A A ‚ A A A A A BA B AAB AAADBACA A ABBBBBBABBDBDBA A A ‚ BAA BA B AADBCAAACC DCCA FA AA A A A ‚ A B ABAB AA B A AADBBC C CC A ‚ A A A A AA A C B BB BABBA A A ‚ A A AA A AA AB A A C A A A -500 ˆ C A BA B AB A A A ‚ A A B ‚ A A AAA A AAA A ‚ AB ‚ ‚ -1000 ˆ Šƒˆƒƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒƒƒƒˆƒƒ 0 250 500 750 1000 1250 1500 1750 2000 Observed NPP, outliers dropped NOTE: 12 obs had missing values. 30 EMDI Outlier Analysis January 28, 2000 226 BIOME Desert Forest Forest / Boreal Forest / Tempera Forest / Tropica Grassland Mediterranean Not specified Savanna Tundra ‚ ‚******** ‚ ‚*************************** ‚ ‚******************* ‚ ‚*********************************** ‚ ‚***************************************** ‚ ‚********************* ‚ ‚************************* ‚ ‚******************************** ‚ ‚*************************** ‚ ‚****** ‚ Šƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒ 100 200 300 400 500 600 700 800 Freq NPP_OLD Mean 10 165.0000 56 533.3185 119 385.7986 379 705.9933 213 812.0187 68 428.0123 5 508.1667 54 632.9198 11 536.6414 11 110.3810 NPP_OLD Mean 31 EMDI Outlier Analysis January 28, 2000 227 BIOMENEW DBL forest / bor DBL forest / tem DBL forest / tro DNL forest / bor EBL forest / tem EBL forest / tro ENL forest / bor ENL forest / tem Savanna / temper Savanna / tropic desert grassland / C3 grassland / C4 t mediterranean mixed forest tundra ‚ ‚********************* ‚****************************** ‚****************************** ‚************************* ‚******************************************* ‚*********************************************** ‚****************** ‚******************************** ‚*********************** ‚**************************** ‚********** ‚****************** ‚************************* ‚*********************** ‚*************************** ‚****** ‚ Šƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒ 100 200 300 400 500 600 700 800 900 Freq Observed NPP, ou Mean 10 199 11 28 215 98 65 139 1 6 17 36 40 9 38 14 415.000 591.856 598.384 497.929 852.693 931.735 351.741 631.509 450.000 551.667 191.765 357.870 496.829 466.667 535.757 123.661 Observed NPP, outliers dropped 32 EMDI Outlier Analysis January 28, 2000 228 BIOMENEW DBL forest / bor DBL forest / tem DBL forest / tro DNL forest / bor EBL forest / tem EBL forest / tro ENL forest / bor ENL forest / tem Savanna / temper Savanna / tropic desert grassland / C3 grassland / C4 t mediterranean mixed forest tundra Difference: Old Freq Mean ‚ ‚ ‚*******************************************************‚ ‚ ‚**** ‚ ‚********* ‚ **‚ ‚ ‚ ‚ ‚* ‚ *****************************‚ ‚ ‚*** ‚ ‚ ‚ ‚ ‚ ********‚ ‚ ***‚ ‚ **‚ ‚ *************‚ ‚ **‚ ‚ ‚* ‚ ‚ Šˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒ -110 -100 -90 -80 -70 -60 -50 -40 -30 -20 -10 0 10 10 199 11 28 215 98 65 139 1 6 17 36 40 9 38 14 -109.3333 8.3305 18.7879 -4.6250 0.7897 2.0700 -57.8172 5.9363 0.0000 0.0000 -16.6176 -6.4842 -3.8194 -25.8333 -3.0835 2.3592 Difference: Old - New NPP 33 EMDI Outlier Analysis January 28, 2000 229 BIOMENEW DBL forest / bor DBL forest / tem DBL forest / tro DNL forest / bor EBL forest / tem EBL forest / tro ENL forest / bor ENL forest / tem Savanna / temper Savanna / tropic desert grassland / C3 grassland / C4 t mediterranean mixed forest tundra Predicted(ensemb Freq Mean ‚ ‚ ‚ ‚*** ‚ ‚*************** ‚ ‚****************** ‚ ‚* ‚ ‚****** ‚ ‚************* ‚ ‚************************ ‚ ‚********* ‚ ‚************************************ ‚************‚ ‚ ‚**** ‚ ‚****************** ‚ ‚******************* ‚ ‚******* ‚ ‚***************************** ‚ ‚********************* ‚ ‚ Šƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒ -60 0 60 120 180 240 9 196 11 28 215 98 65 139 1 6 17 36 38 9 37 9 19.5804 108.7997 133.5819 5.1660 45.9032 94.4948 176.5225 65.6181 269.8430 -91.2120 29.8861 132.1704 141.8768 52.0949 215.1300 160.4227 Predicted(ensemble) - Observed NPP 34