CIRA Activities for NASA ROSES Project
Q4 2008 – June 2009
June 2009
Hardware
1. Replaced the drive bay on Duli's PC to reset the 1.5 TB drive
2. Ordered a web server and UPS unit from RunPC for $4635
a.
b.
c.
d.
e.
f.
g.
h.
i.
j.
k.
l.
m.
n.
o.
p.
q.
CHENBRO SR10569-BK-T TOWER BLK NO PS EATX ROHS CS4521
CHENBRO 84H220910-041 4 SATA HS MODULE FOR SR105/209 BLK CS4527
ZIPPY MRM-6650P 650W EPS12V (24+8) 1+1 REDUNDANT POWER SUPPLY PS1072
INTEL S3210SHLC SKT 775 SATA/DUAL GBE/DDR2/PCIE/PCI MB2188
Four 2GB DDR2-800 ECC UNBUFFERED RM2021
INTEGRATED VIDEO COMPONENT MC9001
INTEGRATED GIGABIT LAN MC9040
INTEGRATED SATA RAID 0, 1, 0+1, 5 MC9028
LSI SAS3081E-R SAS SATA 8T PCIE 8-PORT 300 MBPS 3GB DC1138
8087 TO 4*SATA FORWARD 1.0 M CBL-SFF8087OCF-10M CB0307
BASE SERVER ALTERNATIVE OS BS1401
NOBILIS STANDARD SYSTEM WARRANTY (1 YEAR) WR0018
Six WD WD3000HLFS 300GB SATA 3GBS 16MB 10KRPM W/ 3.5" KIT HD3160
INTEL XEON X3370 3.0GHZ 12M 1333FSB BX80569X3370 CP1371
Two 3.5" HDD MOUNTING BRACKET FOR 5.25" BAY # HD-MOUNTING BR0005
LITEON 24X DVDRW SATA BLACK BULK NO SW #IHAS124-04
POWERCOM KING PRO KIN-2200AP
Data
1.
2.
3.
a.
b.
c.
4.
5.
a.
b.
c.
6.
7.
8.
9.
10.
Created 41000 GIF images (2 GB) of the NASA satellite angstrom exponent from Terra MODIS daily files
Moved all satellite images on Matar to Poudre
USDA NRCS SSURGO STATSGO soils was loaded into ArcSDE and SQL Server.
The data was acquired from http://soildatamart.nrcs.usda.gov/USDGSM.aspx
It contained spatial shapefiles, tabular text, and Microsoft Access MDB files
The MDB files were imported into SQL Server and shapefiles loaded into ArcSDE
Loaded improve, county, state, and national park service spatial data into ArcSDE.
Downloaded TERRA MODIS MOD14 Thermal Anomalies - Fires and Biomass Burning data from USFS. Data acquired from http://modisfire.umd.edu/validation.asp, http://activefiremaps.fs.fed.us/ and http://activefiremaps.fs.fed.us/gisdata.php The spatial fire was loaded
into ArcSDE
Each year spatial layer was different; so, the layers were edited to make the attribute columns identical.
2007 and 2008 had calendar year field called "JULIAN" so added this field in the 2005 and 2006 years using a VB script which can be found
under http://svn.cira.colostate.edu/cira/john/fires.
Created a DAY field for to be consistent across all datasets.
Defined the metadata to be used for the TERRA Level-3 MODIS Atmosphere Daily Global Product and the AURA Ozone Monitoring
Instrument Daily L2 Global 0.25deg Lat/Lon Grid Nitrous Dioxide (NO2) Total & Tropospheric Column and Sulfur Dioxide (SO2) Total
Column datasets. See http://victor.cira.colostate.edu/links/ satellite metadata.
Loaded land use (http://water.usgs.gov/GIS/dsdl/ds240/index.html) data into ArcSDE
Created a shapefile from the 2006 OMNO2G AURA satellite data and loaded over 764,000 feature points into ArcSDE. The following AURA
OMNO2 datasets were loaded into ArcSDE: YYYYMMDD,DAY,TIME,LAT,LONG,SZA,SAA,VZA,VAA,MQF,TP,TR,NO2,BCNO2,CANO2,CF
Created a shapefile from the 2006 TERRA MODIS satellite data and loaded over 437,000 feature points into ArcSDE. The following TERRA
MODIS datasets were loaded into ArcSDE:
YYYYMMDD,DAY,LAT,LONG,SZA,SAA,SENSZA,SENSAA,CF_LIQ,CF_ICE,CF_ALL,O3TOT,AE1OM,AE2OM,AELM,ODRSLM,ODRSOM,ODLOM
Created a SQL Server 2008 spatial database not dependent upon ESRI software and loaded NASA satellite data into it.
Documentation
1. Researched the terminology for NASA fire data. The fire detection data is labeled as MOD14 and MOD14MOD09, the eight day collection
is labeled as MOD14A2.
2. Researched the Microsoft Visual Earth API and SharpMap API.
3. Documented May work and prepared a power point for the Killer App
4. Wrote a Boxocology design report for the spatial data engine (SDE) project containing database, .NET class libraries for data access, web
services, and SOAP exchange for VIEWS and custom prototype browser.
5. Researched how to use Windows Communication Foundation (WCF) web services.
6. Created a specifications document for the AURA OMNO2G datasets to be loaded into CIRA databases.
7. Created a specifications document for the TERRA MODIS Gridded (08) datasets to be loaded into CIRA databases
Software
1. Updated NASA JavaScript application to access and display OMNO2.003 JPEG images
2. Built and developed an application using the SharpMap library. Modified the web application to plot lines and points as well as polygons.
CIRA Activities for NASA ROSES Project
Q4 2008 – June 2009
3.
4.
5.
6.
Wrote a SAT2SHP utility to convert satellite data into spatial data
Created Matlab scripts for the Aura OMTO3G, OMNO2G and OMSO2G satellite data for creating columnar data to load into SQL Server
Refactored the spatial data engine (SDE) interface software to optimize web service access for VML, GML, and KML
Created web services for accessing the ArcSDE spatial data: CIRA Improve, TERRA MODIS MOD14 Thermal Anomalies, AURA OMI NO2G
nitrous dioxide, TERRA MODIS MOD08 optical depth, National Park Service boundaries, NRCS soils, states, counties, and land use. See
http://victor.cira.colostate.edu/Sde-Service
Configuration Management
1. Backed up subversion on Poudre from C: to Poudre's E: partition, and copied files to Victor F:\svn
2. Updated loadshapefiles.sh to include the fire spatial data.
3. Configured Ubuntu server for DHCP name hosting and merging with the CIRA windows nameserver using dhclient
4. Modified the security structure of Windows Server 2008 using GPEDIT.MSC
5. Established subversion on the Ubuntu server
6. Configured iptables on the Ubuntu server to control the INPUT/OUTPUT of port activity
7. Modified apache to only run under port 443 (https)
Health Management
1. Spatial data restore and import both failed when going from SQL 2008 to SQL 2000.
2. Spatial data load using ArcSDE commands under Victor Windows 2008 was successful to Vidi
3. Moved the health monitor to the Ubuntu server
4. Stopped the esri_sde service on Vidi. The last load of data by ArcSDE commands did not release the memory
May 2009
Hardware
3. Replaced the CD reader on Victor with the DVD/CD reader from Bret's old machine (24X CD and 3.2X DVD)
4. Updated the firmware bios on Victor's DVD drive from 1.0b to 1.1a that (finally) enabled DVD reading.
5. Reset Vista to reboot on a power loss
Data
11. Created MDB files containing soils data for the WGA Western states (ak,az,ca,co,hi,id,ks,mt,nd,ne,nm,or,sd,tx,ut,wa)
12. Imported all the MDB files into one SQL Server SDM database
13. Edited the Improve spatial layer and updated the kmz file, see http://vista.cira.colostate.edu/kml
14. Created a new ArcGIS SDE database and loaded 250 MB of soils, land use, county, state, and Improve spatial layers using a script
https://poudre.cira.colostate.edu/airdata/gis/data/loadshapefiles.sh and ArcSDE commands
15. Converted shapefiles from (4) above to layers for exporting out to ArcGIS Server
16. Created KMZ soils files for each western state. See http://matar.cira.colostate.edu/kmz/soils_??.kmz where ?? is the state abbr.
17. Recreated OMAERUVd.003 angstrom exponent images to range from 0-3, MODIS aerosol optical depth images to range from 0-5
18. Created JPEG images using Matlab for OMNO2G.003 ColumnAmountNO2Polluted over Asia and the World
Documentation
8. Completed Automated File Cleanup command utility documentation
9. Researched how to load spatial layers into ArcGIS
10. Research how SAS can provide output formats including OLE DB.
Software
7. Completed VB.NET Automatic File Cleanup suite
a. Windows Service - https://poudre.cira.colostate.edu/airdata_sys/CleanupService
b. Command to use the XML files - https://poudre.cira.colostate.edu/airdata_sys/CleanupUtility
c. Command that uses options - https://poudre.cira.colostate.edu/airdata_sys/Cleanup
8. Added new spatial .NET solutions for accessing the ArcSDE datasets, see https://poudre.cira.colostate.edu/airdata/sde
9. Built the latest GrADS-2.0.a5.oga.5 for Windows and Linux, see http://victor.cira.colostate.edu/links/
10. Modified http://vista.cira.colostate.edu/nasa to include OMNO2G ColumnAmountNO2Polluted images
11. Edited do_omaeruvd_{namer,world}.sh shell scripts to set the range for angstrom exponent images; edited month_ae{namer,world}.sh,
mod08_ae_{namer,world}.sh, and myd08_ae_{namer,world}.sh to set the range for aerosol optical depth.
12. Edited Matlab scripts to produce images consistent with GrADS images including: header, footer, citation, and date.
13. Developed a .NET geospatial web interface to produce spatial data based on a User's selected area of interest
a. Created a spatial data library defining the point and projections that is installed into the global assembly cache using GacUtil
b. Created a spatial data API that connects to the spatial database, transforms and clips data that is installed into component services
using the RegSVCS command utility,
CIRA Activities for NASA ROSES Project
Q4 2008 – June 2009
c.
Created a web service that transforms spatial features to VML, extracts example soils layers from the spatial data engine (sde)
database, and transforms spatial features to GML.
d. Created a prototype browser to display TerraService gateway, imagery, and digital line graph layers and also to allow a user to zoom
and pan to their area of interest and display the soils, land use, and national parks service (NPS) spatial data. See
http://victor.cira.colostate.edu/Sdm-Service/Default.htm
14. Installed and configured ArcGIS Server for use of spatial data on Victor
a. Out of the box install did not work. Directories had to be given special permission for the five GIS users: ArcGISSOM, ArcGISSOC,
argsvruser, arcsvradmin, and ArcGISWebService.
b. The component services had to be modified to allow local activation permission to ArcGISSOM user
c. By itself there is no "user interface" to ArcGIS Server. It is intended as a tool to combine a variety of differing spatial objects under
one umbrella for use by specialized clients like Google Earth or ArcMAP.
d. Application Web interfaces are designed to be generic for either Java or .NET development.
e. KML linking simply put (copied) the KMZ files on your computer to a directory under the ArcGIS Server umbrella. It did not create an
ASP.NET interface to the KMZ files.
f. The ArcSOM and ArcSOC services do not start up when the server is rebooted. They have to be manually started.
g. The help documents leave much to be desired. Most help was found Googling blogs on the issues.
15. Successfully configured the spatial data web service to extract data from the soils tabular "sdm" database
a. The esri_sde service must be stopped before editing any "sde" database table changes; otherwise, a "locked" condition occurs.
Configuration Management
8. Backed up subversion on Poudre from C: to Poudre's E: partition, and copied files to Victor F:\svn
9. Installed WinBind on Ubunt2 to allow the Linux server to join the CIRA Windows Network
10. Edited Apache and Subversion configuration files on Ubunt2 for team collaboration
11. Enabled ArcGIS on Victor using the License file sent from Diane Noren
12. To enable services to run under Windows (Visual Studio) 2008
a. Create a virtual directory to the physical path
b. Add application under IIS
c. Project-ASP.NET Configuration - enable anonymous configuration
d. Properties - handler mappings - enable *.cgi and *.exe as appropriate
13. Enabling the Desktop Experience feature
a. Click Start, click Administrative Tools, and then double-click Server Manager.
b. In Server Manager, click Add Features under Features Summary.
c. In the Add Features Wizard dialog box, make sure that the Desktop Experience option is selected.
d. Click Next, and then click Install.
e. After the installation process is complete, click Close, and then close Server Manager.
14. Enabling the ArcSDE Services and Poudre ArcCatalog to view the Victor ArcSDE spatial layers
a. Opened firewall port 5151 and also port 1433 for SQL Server access
b. Set the SDEHOME environment variable to C:\Program Files (x86)\ArcGIS\ArcSDE
c. started the esri_sde service
d. Edited Poudre C:\Windows\system32\drivers\etc\services to add "esri_sde
5151/tcp" service port
15. Executed sentinel driver 7.5.0 and LMSetup, and configured license manager for use on victor
16. Updated GrADS scripts in https://poudre.cira.colostate.edu/airdata/gifs
17. Created a new Subversion repository on http://ubunt2.cira.colostate.edu/cira for Duli, myself and others. See
http://ubunt2.cira.colostate.edu/matlab for Matlab scripts to create JPEG images
18. Backed up Poudre Subversion to the Linux server. Created a web interface to the datasets including ViewVC.
19. Disabled WinHTTPProxy service on Mater, tightened up security for port 1433, added logging, and cleaned out old Temp files.
Health Management
5. Updated Poudre and Matar Windows Server 2003 and rebooted
6. Assisted Duli with reading HDF5 files into Matlab and suggested improvements to increase Matlab performance
7. Modified Matar bios to change the over clocking to normal operation and to start up after a power outage.
April 2009
Hardware
1. Brought up the new Database server Intel S3200SHC board, Intel SR1530 HSH system, four x3370 processors, two Western Digital
Velociraptor 300 GB 10K RPM drives in RAID0 for the C: Data partition, four Seagate 1.5 TB 7200 RPM drives in RAID0 for the D: Data
2. Replaced Helene’s hard drive from Platte with a 300GB drive, gave old drive to Bill
Data
CIRA Activities for NASA ROSES Project
Q4 2008 – June 2009
1.
Regenerated over 14,000 Monthly MODIS GIF images per Duli Chand's request to standardize the AOD display bar
Documentation
1. Added Automated File Cleanup Utility Word document, see https://poudre.cira.colostate.edu/airdata_sys/afcu
2. Documented how the CMAQ Lambert Conformal Conic images were created using GrADS, see
https://poudre.cira.colostate.edu/airdata/gif/CMAQ/pdef.doc
3. Prepared a power point presentation of current and proposed work for April 24, 2009
4. Documented security changes required to run CIS on Helene's machine.
a. relax the UAC, see http://www.petri.co.il/disable_uac_in_windows_vista.htm
b. adjust the IE Tools-Options-Security-Trusted Sites to add busfin, wwwlb, and wsprod colostate servers trusted sites;
c. adjust the IE Tools-Options-Security-Custom level for trusted sites to allow ActiveX;
d. install the Jinitiator, the current version is 1.3.1.30 but the older 1.3.1.18 works;
e. copy the C:\Program Files\Java\jre\bin\client\bin\jvm.dll to the C:\Program Files\Oracle\JInitiator 1.3.1.18\bin\hotspot
directory
5. Compared two VB.Net User Forum Software Packages: InstantForum and BBDotNet
Software
1.
2.
3.
4.
5.
6.
7.
8.
Completed VB.NET Automatic File Cleanup Utility, see https://poudre.cira.colostate.edu/airdata_sys/CleanupService
Built a Windows Service to run Automatic File Cleanup Utility (AFCU) application
Modified the shell scripts used to generate the GrADS monthly MODIS images
Created a VB.Net Windows service to perform Automatic File Cleanup
Built a .NET NetCDF 4.0 DLL, Include, and Library files for Bret, see http://victor.cira.colostate.edu/links/netcdf4.zip
Built a .NET Google Earth application to display the HYSPLIT model data
Modified AFCU Windows Service per standards
Added a Cleanup Utility command line application project to the AFCU solution
Configuration Management
1. Backed up subversion on Poudre from C: to Poudre's E: partition, and copied files to Victor F:\svn
2. Installed and configured SQL Server on Virtual Air Data Exchange Repository "Vader", the new Database Server
3. Backed up the AIRDATA_DSS database from Vidi to Victor
4. Backed up the AIRDATA_CORE from Viper to Victor
5. Installed ESRI ArcGIS Server and ArcSDE on Victor and Sentinel 7.5 driver for the dongle
6. Added the ServiceInstaller source code to https://poudre.cira.colostate.edu/airdata_sys/ServiceInstaller
7. Installed Ubuntu O/S, multiple utilities, NetCDF and HDF on Ubunt2, 129.82.109.39
Health Management
1. Restart Poudre sshd daemon on two separate days to allow Big Brother data transfer from Monitor
2. Removed 27 viruses from Helene's XP PC
3. Rebuilt Helene's Vista PC, loaded application software, and configured applications, sent email to Josh Clark
4. Assisted Duli with software installation and use and computer operation
5. Downloaded AVG anti-virus version 8.5 and cut to CD. Installed on Helene's PC.
6. Downloaded Ubuntu Linux operating system version 8.04 server and cut to CD
7. Power loss over 4/18-19 weekend caused a loss of time of two hours on the 20th
8. Removed Helene's hard drive from Platte, replaced it with a WD 320GB drive, and installed Linux Ubuntu 8.04
March 2009
Hardware
1. Brought up the new Database server Intel S3200SHC board, Intel SR1530 HSH system, four x3370 processors, two Western Digital
Velociraptor 300 GB 10K RPM drives in RAID0 for the C: Data partition, four Seagate 1.5 TB 7200 RPM drives in RAID0 for the D: Data
partition. Control-C set the LSI card for the Seagate drives RAID0. Control-I set the Intel RAID0 for the Western Digital drives. During
installation of the win2K8 OS the 558 GB were formatted with the TB drives unallocated. The Win2K8 ISO DVD was created on Poudre
from the E: partition using Nero. After the Win2K8 OS was installed on the C: Vader partition, the 5.7 TB D: Data partition was set to
online and the property was set to GPT. The D: data partition was formatted with 64K allocation units.
2. Ordered and received desktop PC for Duli, INTEL DQ35JOE CORE2 DUO E8400 1333B/6M, 3.0 GHz, 2 GB DDR2 800 Ram, 320 MB hard disk
3. Requested quote from RunPC for two UPS battery replacements.
4. Worked on hardware drivers for Victor
Data
1.
Created National Parks KMZ file from shapefiles using ArcGIS 9.3.
CIRA Activities for NASA ROSES Project
Q4 2008 – June 2009
2.
3.
4.
5.
6.
7.
8.
Downloaded images for Calipso Caliop data from NASA and put them on Matar.
Converted Hysplit output files to KML files. There were 8928 files per CSV files. Put the files out under
http://matar.cira.colostate.edu/trajectory/Product/Year
Created a background for the webcams display and webcams data. Converted pX and pY data to UTMx and UTMy and then to Latitude
Longitude. Took the data, Webcam Name, and URI and created a shapefile. Used Poudre to convert the shapefile to a KML file.
Created shapefile and KMZ files for ARS CAM Locations; see http://vista.cira.colostate.edu/kml/
Created over 1000 new daily GIF images for Angstrom Exponent in 2006
Created 4320 images of 16 monthly MOD08_M3.005 MODIS TERRA and MYD08_M3.051 Aqua satellite parameters of optical depth and
angstrom exponent for 2006 and 2007
Regenerated nearly 200,000 Aura (9.6GB), Terra (1.4GB), Aqua (2.4GB) satellite GIF images based on analysis of Duli Chand
Documentation
1. Added https://poudre.cira.colostate.edu/airdata/gis/documents/Webcams.doc describing the procedure to transform Lambert
Conformal Conic data into latitude-longitude WGS84 data.
2. Documented the incidence of SQL select statements for airfact3_rhr in AppFramework.Chartv2.
Software
1. Updated CiraGeoKml Google Map software to interface and to load National Park Google Earth KMZ files. See the Url
http://vista.cira.colostate.edu/kml
2. Created a web interface to Caliop data at http://matar.cira.colostate.edu/calipso/
3. Created Google Maps JavaScript utility to display the Hysplit KML files. See http://vista.cira.colostate.edu/Hysplit.
4. Created a web page to display the webcams. Created scripts to process the Lambert Conformal Conic UTM data into WGS84 latlong.
Updated http://vista.cira.colostate.edu/kml/ to include the WebcamURI.kmz Google Earth KML "webcams" file.
5. Built Composition subset of AnnualSummary
6. Created scripts to process NASA S4PA Terra MODIS data for Angstrom Exponent. Updated NASA prototype to display images
7. Created http://matar.cira.colostate.edu/modis to display monthly MODIS NASA satellite data
8. Built http://nco.sf.net NetCDF Operators for Windows, see http://victor.cira.colostate.edu/links/nco-3.9.7.zip
9. Built VB.NET Automatic File Cleanup Utility, see https://poudre.cira.colostate.edu/airdata_sys/afcu
Configuration Management
1. Backed up subversion on Poudre from C: to Poudre's E: partition, and copied files to Victor F:\svn
2. Investigate the IIS Logging in order to capture IP Addresses for VIEWS users’ recognition. Created web logs of Vista for 2003 through 2009
and web logs of Matar for 2005 through 2009, see http://matar.cira.colostate.edu/WebLogs
3. Downloaded GrADS 2.0a5 software and configured it for use on Victor. Updated http://victor.cira.colostate.edu/links
4. Investigated the cost for Google Earth Enterprise solution; however the cost of $115,000 was too high.
5. Created new scripts to create daily and monthly images, see https://poudre.cira.colostate.edu/airdata/gif
6. Assisted Duli in loading GrADS, HDF utilities, and related software on his new PC
7. Changed Matar IIS virtual directory s4pa to point to the E:\s4pa. The C: partition was over 75% full.
8. Edited Links page for NetCDF Operators ZIP file link.
Health Management
1. Brought up the servers after the power outage on March 3rd. Removed ssl_mutex to start Poudre subversion system. Changed order of
boot devices for Matar, Viper and Viscount. Manually booted Matar. Lost time was two hours. Repeated process for a planned outage.
Lost power again on March 22nd and Matar went down without coming back up. Changed Microsoft update on Matar to a manual
process. After a reboot, Matar failed to come up.
2. Created a new monitoring service for CIRA servers, see https://poudre.cira.colostate.edu/bb/
3. Assisted Duli in System Remote File sharing, Firewall configuration, and User configuration
4. Added round robin database (RRD) data storage to Big Brother monitoring for PNG graphics display
February 2009
Hardware
1. Purchased new database server, Intel 3210 motherboard with XEON X3370 3.0GHZ 12M 1333FSB quad core processor, four DDR2-800
2MB RAM modules, two WD 300 GB 16MB cache drives for OS, and four Seagate 1.5 TB 32MB cache drives in RAID0 for the database.
2. Purchased two additional 1.5TB hard drives and a Powercom King Pro UPS
Data
1.
Downloaded Aura_OMI_Level3 Products as shown in the table, converted 2004-2009 datasets from HE5 to HDF, and created over
105,000 GIF images over North America using GrADS on Victor. See http://vista.cira.colostate.edu/nasa for a prototype tool that displays
the images. See the URL https://poudre.cira.colostate.edu/airdata/gif area for the scripts used to create the graphic images.
CIRA Activities for NASA ROSES Project
Q4 2008 – June 2009
OMAERUVd.003
FinalAerosolAbsOpticalDepth388
FinalAerosolAbsOpticalDepth500
FinalAerosolExtOpticalDepth388
FinalAerosolExtOpticalDepth500
2.
3.
4.
OMAEROe003
AbsorbingAerosolOpticalThicknessMW
AerosolModelMW
AerosolOpticalThicknessMW
AerosolOpticalThicknessPassedThresholdMean
AerosolOpticalThicknessPassedThresholdStd
SingleScatteringAlbedoMW
SingleScatteringAlbedoPassedThresholdMean
SingleScatteringAlbedoPassedThresholdStd
SolarZenithAngle
TerrainReflectivity
UVAerosolIndex
VISAerosolIndex
ViewingZenithAngle
OMTO3e.003
Column Amount Ozone
Radiative Cloud Fraction
Solar Zenith Angle
Viewing Zenith Angle
OMDOAO3e
Cloud Fraction
Cloud Fraction Precision
Cloud Pressure
Cloud Pressure Precision
Column Amount Ozone
Column Amt Ozone Prec
Effective Temperature
Eff. Temperature Precision
Ghost Column Amount O3
Solar Zenith Angle
Terrain Reflectivity
Viewing Zenith Angle
Converted NetCDF Terra and Aqua MOD04_CMAQ and GASP CMAQ Products into 15000 GIF images using GrADS. See the URL
https://poudre.cira.colostate.edu/airdata/gif/CMAQ/ for the scripts used to perform the conversion.
Downloaded Aura TES TL3O3d (Ozone) and TL3COD (Carbon Monoxide) Products and converted HDF files into 5000 GIF images using
GrADS. See the URL http://vista.cira.colostate.edu/tes to display the images. See the URL https://poudre.cira.colostate.edu/airdata/gif/
TES-Aura_L3/ for the scripts used to perform the conversion.
Downloaded state and county shapefiles, converted them to Google Earth KMZ files, and put them out on the Matar server.
a. county) each state's two character name ending with .kmz
b. state) there are four: Lower48_States.kmz, PuertoRico.kmz, Alaska.kmz, and Hawaii.kmz
c. combo) Lower48_Counties.kmz has all the lower48 counties
d. nps) National_Parks_Boundaries.kmz has the BTS 2006 boundaries
Documentation
1. Added documentation for the NASA satellite data to https://poudre.cira.colostate.edu/airdata/doc including a top level directory.xml as
well as individual directories for each CMAQ, TES, and S4PA NASA data product.
2. Added the new server hardware configuration to https://poudre.cira.colostate.edu/airdata_sys/hardware
3. Diagnosed the differences between the Aura_OMI_Level2G and Aura_OMI_Level3 and added word documents describing all the NASA
data products to https://poudre.cira.colostate.edu/airdata/doc in their separate data product directories.
4. Level3 Aura NASA data have different missing values, some datasets have a third dimension, and grid sizes can vary from product to
product. The (1,1) cell is (-89.875,-179.875) or (-89.5,-179.5) respective to grid sizes of 0.25 or 1.0 degree in size. The Latitude and
Longitude grids are consistent. Missing values are either -1.2676506E30, -32767, or 65535.
5. Level2G Aura NASA data each have their own Time, Latitude, and Longitude grids. All grids are sparsely populated. The dataset cell has a
corresponding latitude, longitude, and time in an associated grid. The (1,1)cell does not correspond to any specific location on the earth.
The Latitude and Longitude grids contain the values respectively when the OMI sensor collected the data. The fill value and missing value
are usually -1.2676506E30.
Software
1. Updated NASA JavaScript to display the CMAQ and Aura satellite GIF images and updated http://vista.cira.colostate.edu/nasa
2. Created a new TES JavaScript page to display the Aura TES satellite data on http://vista.cira.colostate.edu/tes
3. Built HDF5 version 1.8.2 in VS2008 .NET and developed two utilities to parse NASA HE5 data for insertion of Aura data into the VIEWS SQL
Server database
a. He5ls
b. He5todata
4. Created scripts to create the North America and worldwide NASA Aura OMAEROe, OMAERUVd, OMDOAO3e, OMLER, and OMTO3e
Product images using GrADS and H5toH4. See https://poudre.cira.colostate.edu/airdata/gif/Aura_OMI_Level3
5. Created scripts to create the CMAQ images for North America of MODIS and GASP data overlay on AOD grid. See
https://poudre.cira.colostate.edu/airdata/gif/CMAQ
6. Created a .NET solution for CiraGeoKml to load and demonstrate the Google Earth KMZ files. See the Url
http://vista.cira.colostate.edu/kml
Configuration Management
1. Backed up subversion daily on Poudre to Victor, backed up C: to its E: drive on Matar
2. Redesigned the directory structure of Matar images to accommodate regions in the world such as North America. Updated NASA
JavaScript accordingly to Subversion on Poudre.
3. Updated installation software under http://victor.cira.colostate.edu/links/index.html on Victor
CIRA Activities for NASA ROSES Project
Q4 2008 – June 2009
4.
Added a new area “gif” under subversion and posted the scripts, control, and run files used to create GrADS images on Poudre to the
https://poudre.cira.colostate.edu/airdata/gif directory on Poudre.
Health Management
5. Installed AVG 8.0 on Matar, Poudre, Victor, Keid, Viscount, Viper Windows 2003 and 2008 servers
6. Rebooted all systems Friday morning of the 20th after a double electrical outage.
January 2009
Data
1. Downloaded Aura_OMI_Level2G and L3 sample data Victor
2. Used above sample data files to create XML and XSD files for each of the satellite types
Design
1. Determined storage requirements for individual satellite data files of a specified format
2. Estimated storage requirements for ongoing download and update of relevant satellite data
3. Estimated download and update frequency for specified satellite datasets
Documentation
1. Added Terra, Aqua, Aura and Calipso satellite documentation to https://poudre.cira.colostate.edu/airdata/doc
Software
1. Built and added HDF5, and HDF5Tools to https://poudre.cira.colostate.edu/airdata/hdf
2. Added IDL files for Calipso to https://poudre.cira.colostate.edu/airdata/idl
3. Started developing a tool (he5todata) to output satellite data for input into SQL Server
4. Updated NASA JavaScript prototype tool and put under http://vista.cira.colostate.edu/nasa
December 2008
Data
1. Downloaded Aqua_MODIS_Level3 MYD08_D3.005 satellite data for 2006 on Victor
2. Created GIF images of Aqua MODIS Level3 satellite data using GrADS for COT, COD, MCO on Matar
3. Downloaded Aura_OMI_Level2G data for OMAEROG 2006 data on Victor
Design
1. Identified candidate fields in raw satellite data files for possible import into the VIEWS RDBMS
2. Designed a prototype schema for the candidate fields
3. Examined the existing RDBMS schema for necessary modifications in order to accommodate the satellite data schema
Documentation
1. Added Aura satellite data documentation to https://poudre.cira.colostate.edu/airdata/hdf/documents
Software
1. Updated NASA JavaScript to display the Aqua satellite GIF images and added new Presets
2. Downloaded HDF5 version 1.8.2 and built it for development
3. Installed Active Perl on Poudre for cgi-bin Perl script execution
4. Installed and configured HDF5-1.8.2, SZLIB-2.1, ZLIB-1.2.3, and JPEG6 for building HDF5 utilities
5. Built OpenDAP 3.8.2, LIBOC-DAP, and GrADS to display Aura satellite data
6. Built a H5toH4 utility to convert Aura satellite data to HDF4 for processing with GrADS on Victor
7. Built the src2shp utility under VS2008 DotNet for creating shapefiles from Views data
8. Built a DotNet wrapper for the shapelib unmanaged code, https://poudre.cira.colostate.edu/viewvc/airdata/gis/ShpLib/
Configuration Management
1. Added HDFLOOK satellite data reader and updated NASA JavaScript to Subversion on Victor
2. Added the H5toH4 utility to cira_sys on Poudre
3. Located all SVN, BB, GrADS software under http://victor/svn_bb_tools/index.html
November 2008
Hardware
1. Added two 300GB drives on Victor
2. Configured and formatted the two 300 GB drives for RAID as E: partition on Victor
3. Formatted the one 140 GB drive for Backup as F: on Victor
CIRA Activities for NASA ROSES Project
Q4 2008 – June 2009
4.
Installed SATA 378 fasttx2k.sys driver on Matar
1.
2.
3.
4.
Installed ArcSDE 9.2 on Victor
Downloaded Land Use shapefiles from USGS and merged them all into one layer on Poudre
Loaded the Land Use layer into ArcIMS on Viscount
Analyzed the NASA satellite data for graphics display on Victor:
a. Terra_MODIS_Level3
b. Aqua_MODIS_Level3
c. Aura_OMI_Level2G
Downloaded 2006 Terra_MODIS_Level3 data, 30.3 GB on Poudre E:\ drive
Made graphics for 2006 and 2008 NASA satellite data, 20 minutes for one parameter
a. Corrected_Optical_Depth_Land
b. Mass_Concentration_Land
c. Cloud_Optical_Thickness
Data
5.
6.
Software
1. Installed ArcGIS Desktop 9.2 on Victor
2. Created a new src2shp utility for Views to download shapefiles on Victor
3. Put up nasa, maf, and main Google API pages on victor.cira.colostate.edu
4. Installed ArcIMS Server and .NET SDK on Viscount
5. Wrote script to automatically create CTL and GS GrADS for NASA data
Configuration Management
1. Backed up C: onto Victor F: on Victor
2. Installed Subversion and configured svn-access files on Poudre
3. Loaded Annual Summary and GIS documents into SVN on Poudre
4. Sent SVN documentation Knaff/Hughes via email
5. Instantiated a new system repository and loaded apache subversion on Poudre
6. Documented CIRA use of subversion with a cheat sheet on Poudre
7. Backed up the two subversion repositories from C: to E: on Poudre
8. Added SRC2SHP, ShpLib, HDF4, and HDF5 into Subversion from Victor
October 2008
Hardware
1. Inventory CIRA hardware
2. Analyzed alternative RAID scenarios
3. Designed architecture for Database server
a. RAID0 for Windows server 2008
b. RAID0 for logging
c. RAID0 for Data
4. Specifications for three Western Digital 300GB VelociRaptor SATA 3Gb s 3.5" 300 GB hard drives drives for the database files