SCD User Briefing The Community Data Portal and the Earth System Grid 85th AMS Annual Meeting January 9-14, 2005 Don Middleton with presentation material developed by Luca Cinquini, Mary Haley, Fred Clare - and many project collaborators NCAR Scientific Computing Division Section Head, Visualization & Enabling Technologies January 11, 2005 NCAR Scientific Computing Division Supercomputing • Communications • Data Data Organized by… Org Chart (?) NCAR Scientific Computing Division Supercomputing • Communications • Data The NCAR Community Data Portal (CDP) http://cdp.ucar.edu/ QuickTime™ and a TIFF (Uncompressed) decom press or are needed to s ee this picture. QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture. NCAR Scientific Computing Division Supercomputing • Communications • Data CDP Goals Build the cyberinfrastructure for the integration and support of a broad range of geo-informatic projects within NCAR, thus reducing startup cost and development time Develop unified gateway to the large, diverse UCAR/NCAR/UOP data holdings, providing a wide range of data services on these holdings: publishing, browsing, search and discovery, download, remote access, analysis, visualization NCAR Scientific Computing Division Supercomputing • Communications • Data The Community Data Portal - A Brief Tour NCAR Scientific Computing Division Supercomputing • Communications • Data Browse Click NCAR Scientific Computing Division Supercomputing • Communications • Data Mozart Metadata NCAR Scientific Computing Division Supercomputing • Communications • Data Mozart Data NCAR Scientific Computing Division Supercomputing • Communications • Data Search for NCEP ETA Type NCAR Scientific Computing Division Supercomputing • Communications • Data Search for NCEP ETA results NCAR Scientific Computing Division Supercomputing • Communications • Data Search for tree ring Type NCAR Scientific Computing Division Supercomputing • Communications • Data Search for tree ring results NCAR Scientific Computing Division Supercomputing • Communications • Data Advanced Search 1 NCAR Scientific Computing Division Supercomputing • Communications • Data Advanced Search 2 NCAR Scientific Computing Division Supercomputing • Communications • Data Forest Fire Visualization Click NCAR Scientific Computing Division Supercomputing • Communications • Data CDP Home with WACCM Click NCAR Scientific Computing Division Supercomputing • Communications • Data WACCM Data NCAR Scientific Computing Division Supercomputing • Communications • Data WACCM Source Code NCAR Scientific Computing Division Supercomputing • Communications • Data CDP Collaborations and Acknowledgements SCD/DSG: thanks for supporting the hardware! SCD/DSS: metadata and data services SCD/MSS: online access to MSS ESG (including CGD, HAO): shared development, hosting environment, technologies Unidata: joint development of NcML, collaborated on THREDDS search and discovery metadata DLESE, BADC, GCMD: export or exchange (via OAI) of metadata documents for cross-institutional searches COLA: provide remote data services through GRADS Many data providers across UCAR/NCAR/UOP and others: ACD, ATD, CGD (CAS, PCM, CCSM), JOSS, SCD (DSS, VETS), Unidata, WACCM and CU/ENLIL GridBGC: shared development GIS: NetCDF to GIS conversion services BADC: cross-center data discovery ESP: sharing information and technologies NCAR Scientific Computing Division NOMADS: exploratory collaboration Supercomputingundergoing • Communications • Data Climate Data Click NCAR Scientific Computing Division Supercomputing • Communications • Data The Earth System Grid http://www.earthsystemgrid.org NCAR Scientific Computing Division Supercomputing • Communications • Data The ESG Collaboration LBNL: Climate storage facility ANL: Computational grids, & grid-based applications LLNL: Model diagnostics & inter-comparison USC/ISI: Computational grids, & grid-based applications NCAR: Climate change predication and scenarios LANL: Next generation coupled models & computing NCAR Scientific Computing Division Supercomputing • Communications • Data ORNL: Climate storage & computational resources ESG Home NCAR Scientific Computing Division Supercomputing • Communications • Data PCM Metadata NCAR Scientific Computing Division Supercomputing • Communications • Data PCM Files and MSS NCAR Scientific Computing Division Supercomputing • Communications • Data ESG CCSM NCAR Scientific Computing Division Supercomputing • Communications • Data CCSM Datasets NCAR Scientific Computing Division Supercomputing • Communications • Data Subsetting List NCAR Scientific Computing Division Supercomputing • Communications • Data Subsetting Interface NCAR Scientific Computing Division Supercomputing • Communications • Data IPCC Data Click NCAR Scientific Computing Division Supercomputing • Communications • Data Data Available via ESG Portal (November 2004) Community Climate System Model 28.4 Terabytes, including 21 simulations, 141 datasets, and 289,374 files Parallel Climate Model 20.42 Terabytes, including 98 simulations, 434 datasets, and 44,000 files Total 48.8 Terabytes, 119 simulations, 575 datasets, in over 333,872 files 167 registrations, 132 approved, 154.2GB downloaded to date Plus new IPCC Data 150 user registrations, 1.1TB of data downloaded, in 16,000 files NCAR Scientific Computing Division Supercomputing • Communications • Data www.earthsystemgrid.org NCAR Scientific Computing Division Supercomputing • Communications • Data ‘ing Our Data NCAR Scientific Computing Division Supercomputing • Communications • Data Data->Knowledge Mass Storage System (2.0PB) Establish new paradigms for managing and accessing scientific data based on semantic organization. NCAR Scientific Computing Division Supercomputing • Communications • Data Petascale Knowledge Environment END NCAR Scientific Computing Division Supercomputing • Communications • Data