Installation Instructions for the NASA International MODIS/AIRS Processing Package
(IMAPP) RealEarth Web Mapping Service for Direct Broadcast
University of Wisconsin-Madison, Space Science and Engineering Center (SSEC)
Beta Version - 13 March 2014
Table of Contents
Section 1: Introduction .................................................................................................................... 3 1.1 Overview of IMAPP RealEarth ........................................................................................................................ 3 1.2 Requirements ........................................................................................................................................................ 3 1.2 Input Data Requirements ................................................................................................................................. 3 1.3 Disclaimer ............................................................................................................................................................... 4 Section 2: Installation and Configuration ................................................................................ 4 2.1 Overview .................................................................................................................................................................. 4 2.2 IMAPP RealEarth Conceptual Diagram ....................................................................................................... 4 2.2 Implementation Notes ....................................................................................................................................... 5 2.3 Installation of the Virtual Machine ............................................................................................................... 6 2.5 Upgrading the IMAPP RealEarth Virtual Machine ................................................................................. 8 2.6 Removing the IMAPP RealEarth Virtual Machine .................................................................................. 8 Section 3: Displaying IMAPP Products in the IMAPP RealEarth Viewer ........................ 8 3.1 Default Products ................................................................................................................................................... 8 Section 4: Product Manager for IMAPP-­‐RealEarth .............................................................. 11 4.1 Using your Product Manager ....................................................................................................................... 11 Section 5: Using the IMAPP RealEarth Interface .................................................................. 13 5.1 Basic Visualization ............................................................................................................................................ 14 5.2 Composites and Animation ........................................................................................................................... 15 5.3 Keyboard Shortcuts and Animation Control ......................................................................................... 15 5.4 Sharing ................................................................................................................................................................... 15 5.5 Settings .................................................................................................................................................................. 15 Section 6: Additional Features .................................................................................................... 16 6.1 Uploading Other Files ...................................................................................................................................... 16 Section 7: Trouble Shooting ......................................................................................................... 17 7.1 IMAPP RealEarth Installation ...................................................................................................................... 17 7.2 IMAPP RealEarth Products ........................................................................................................................... 17 7.3 IMAPP RealEarth Virtual Machine ............................................................................................................ 17 2
Section 1: Introduction
1.1 Overview of IMAPP RealEarth
IMAPP RealEarth is a browser-based GeoTIFF imagery viewer powered by SSEC’s RealEarth
Web Mapping System (http://wms.ssec.wisc.edu). It enables users to display satellite imagery
in a Google Maps or Google Earth interface. It is distributed as a virtual machine (VM) that is
capable of displaying MODIS (and VIIRS) GeoTIFF images created from the IMAPP/CSPP
Polar2Grid software package in support of Direct Broadcast users. IMAPP RealEarth allows
users to process imagery generated by their IMAPP installation for display in Google Maps, and
share that imagery with users through a web mapping service. Image processing can run in
either a manual or automated mode. This document provides installation instructions, interface
features, trouble shooting and also serves as a users guide.
1.2 Requirements
The following define the host system requirements. Please note: Your system will not be able
to support the IMAPP RealEarth Virtual Machine (VM) unless it meets all of these requirements:
• Intel or AMD CPU with 64-bit instruction support
• RedHat or CentOS Linux v.6 host server
• Root privileges on host server
• Hardware virtualization enabled in the host system BIOS
Host resources that are to be made available to the Virtual Machine (VM) which is released as
an 8 GB file:
• 2 or more CPU cores
• 2GB or more of RAM
• 100 GB of disk space
1.2 Input Data Requirements
IMAPP RealEarth enables a web browser to display, layer, and animate products created using
the IMAPP/CSPP Polar2Grid software. The Polar2grid software creates spatially reprojected
GeoTIFF images from input MODIS Level 1B and VIIRS SDR data. It is freely distributed
through this website:
http://cimss.ssec.wisc.edu/imapp/modis_polar2grid_v1.1.shtml
Users can also use the IMAPP RealEarth system to display their own GeoTIFF files.
Instructions for displaying these files are provided in Section 6.
3
1.3 Disclaimer
Original scripts and automation included as part of this package are distributed under the GNU
GENERAL PUBLIC LICENSE agreement version 3. Binary executable files included as part of
this software package are copyrighted and licensed by their respective organizations, and
distributed consistent with their licensing terms.
The University of Wisconsin-Madison Space Science and Engineering Center (SSEC) makes no
warranty of any kind with regard to the IMAPP RealEarth software or any accompanying
documentation, including but not limited to the implied warranties of merchantability and fitness
for a particular purpose. SSEC does not indemnify any infringement of copyright, patent, or
trademark through the use or modification of this software.
There is no expressed or implied warranty made to anyone as to the suitability of this software
for any purpose. All risk of use is assumed by the user. Users agree not to hold SSEC, the
University of Wisconsin-Madison, or any of its employees or assigns liable for any
consequences resulting from the use of the IMAPP RealEarth software.
Section 2: Installation and Configuration
2.1 Overview
The IMAPP RealEarth Virtual Machine includes a Linux CentOS operating system and software
stack that prepares and manages imagery for IMAPP RealEarth. It is a self-contained system
that requires allocation of host machine resources. The VM itself is distributed as a single 8.0
GB file.
2.2 IMAPP RealEarth Conceptual Diagram
The following diagram shows how the IMAPP RealEarth Virtual Machine (VM) resides within a
host server. In this example, IMAPP Polar2Grid GeoTIFF imagery is generated on a different
machine, uploaded to the VM, and processed so that it can be viewed in a web browser. The
IMAPP Polar2Grid software could also reside on the “Host Server.”
4
IMAPP RealEarth system schematic diagram.
2.2 Implementation Notes
Before beginning the IMAPP RealEarth installation process, it is important to make sure that
Security-Enhanced Linux is not currently enabled on your system. To check for this, type the
following on your linux host machine:
sestatus
If your system returns the following message:
SELinux status:
enabled
please contact your system administrator to see if this can be disabled. The IMAPP RealEarth
VM has not been tested while this software is enabled.
5
2.3 Installation of the Virtual Machine
The first step in the installation process is to prepare the host machine for the Virtual Machine
(VM). The Kernal Virtual Machine (KVM) software and support utilities must be installed.
Executing the following command will check your current system for the required utilities and
then download, update and install them if necessary. Before you begin installation, we suggest
you consult and gain approval from your host machine system administrator. Note, to execute
these commands the user must have root privileges.
sudo –u root -s
yum install kvm virt-manager libvirt libvirt-python python-virtinst
libguestfs libguestfs-tools nfs-utils wget
Next download the VM image file (8 GB) and installation script. The VM image file will need to
remain present in its original location after installation, so be sure to download or copy it to a
location where it can reside and run going forward.
wget http://wms.ssec.wisc.edu:8000/imapp/SSEC-RealEarth-IMAPP.img
wget http://wms.ssec.wisc.edu:8000/imapp/install_realearth-imapp.sh
To execute the script, make sure the permissions allow execution:
chmod guo+x install_realearth-imapp.sh
Install the VM:
./install_realearth-imapp.sh SSEC-RealEarth-IMAPP.img
Answer the series of on-screen questions to configure the VM for your implementation. The
default values are the minimum requirements for an IMAPP RealEarth installation, and can be
used by hitting return without typing in any additional information but may be modified to fit your
preferences and environment. Increasing the number of CPUs and/or memory allocation will
likely improve performance. The configuration options that are presented when running the
installation script are described below.
The IMAPP RealEarth Virtual Machine configuration options are:
• Name of the VM (default: imapp)
• Email (default: none. optional - to receive notifications from Nagios monitoring software
-- such as disk-full or high load)
• Number of CPUs to allocate to the VM (default: 2)
• Amount of Memory to allocate to the VM in MB (default: 2048)
• Enter the port number on the host that will forward www access to the VM (default:
8001)
• Enter the NFS mount point on the host for product data on the VM (default: /data/imapp)
• Enter the latitude and longitude of your location or antenna site (No default).
6
Set-up will complete. This may take a minute. When complete, your VM is now running.
If successful, the VM will exist in the installation directory with the name you chose for the VM
plus a .img extension (example: imapp.img).
To confirm that your Virtual Machine has been successfully installed, open a web browser on a
different computer than the host machine. Navigate to your host machine and the port selected
for forwarding (e.g. http://hostmachine.your.domain:8001 ).
You should see the IMAPP RealEarth controls and a familiar Google Maps interface. An
example of an installation centered over Madison, Wisconsin is shown below.
If your installation was not successful, please refer to the Troubleshooting section at the end of
the document.
7
2.5 Upgrading the IMAPP RealEarth Virtual Machine
When a new Virtual Machine and installation script is available, upgrading the IMAPP RealEarth
VM can be accomplished using the installation script with the -d option:
sudo sh ./install_realearth-imapp.sh -d
The installation script will compare the VM on your host machine with the current version
available, then will download and install a newer version if necessary. Because it is possible to
run multiple IMAPP RealEarth VMs on your host machine simultaneously, you will be presented
with a list of existing VMs and prompted to choose one to upgrade. Follow the instructions
presented by the script to complete your upgrade. Repeat this process to upgrade multiple
VMs.
2.6 Removing the IMAPP RealEarth Virtual Machine
To uninstall the IMAPP RealEarth VM, use the installation script with the -u option:
sudo sh ./install_realearth-imapp.sh -u
Executing this command will present you with a list of VMs currently installed on your host machine.
When you type the name of the VM you wish to uninstall, the script will execute commands that will
shutdown the VM and remove all supporting files. The 8GB source image file (the .img file that you
used to install the VM) will remain on your host machine. This allows you to reinstall it at a later
date or remove the image file all together.
Section 3: Displaying IMAPP Products in the IMAPP RealEarth Viewer
3.1 Default Products
By default, the Virtual Machine (VM) comes ready with 47 predefined views prepared for display of
Polar2Grid MODIS (and VIIRS) GeoTIFF image products. The following table shows a list of the
products predefined in the VM.
Terra and Aqua
VIIRS Spectral Bands
MODIS Spectral Bands
Daytime
True Color RGB
True Color RGB
single pass display
Band 1 Visible
Day/Night Band
and multi-pass composite
(.68 µm reflectances)
I-Band 1 Visible
Band 7 Near Infrared
(.68 µm reflectances)
(2.1 µm reflectances)
I-Band 2 Visible
8
Nighttime
single pass display
and multi-pass composite
Band 20 Short Wave Infrared
(.86 µm reflectances)
(3.7 µm BrightnessTemperatures) I-Band 3 Near Infrared
Band 27 Water Vapor Infrared
(1.6 µm reflectances)
(6.7 µm BrightnessTemperatures) I-Band 4 Short Wave Infrared
Band 11 Long Wave Infrared
(3.7 µm BrightnessTemperatures)
(11 µm Brightness Temperatures) I-Band 5 Long Wave Infrared
(11 µm Brightness Temperatures)IBand 20 Short Wave Infrared
Day/Night Band
(3.7 µm BrightnessTemperatures) I-Band 4 Short Wave Infrared
Band 27 Water Vapor Infrared
(3.7 µm BrightnessTemperatures)
(6.7 µm BrightnessTemperatures) I-Band 5 Long Wave Infrared
Band 11 Long Wave Infrared
(11 µm Brightness Temperatures)I(11 µm Brightness Temperatures)
The Polar2Grid GeoTIFF files need to be copied to the VM using the IMAPP RealEarth upload
scripts. These two scripts, “imapp-upload” and “wmsupload”, are distributed with the VM and
work together to copy IMAPP products for viewing in the Google Maps interface on the VM. They
need to run on the machine where the Polar2Grid GeoTIFF files are produced or are stored. To
implement these scripts, copy them from the VM to the machine where your GeoTIFF files exist:
wget http://hostmachine.your.domain:8001/imapp-upload
wget http://hostmachine.your.domain:8001/wmsupload
Make sure both scripts are executable (e.g. chmod guo+x imapp-upload wmsupload) and are
accessible in your $PATH. Then, when IMAPP’s Polar2Grid creates a GeoTIFF, pass it to the
imapp-upload script to send it to your IMAPP-RealEarth VM. For example, typing this command,
imapp-upload aqua_modis_infrared_20_20140303_185300_wgs84_fit.tif
will upload the Aqua MODIS band 20 (4 micron) GeoTIFF file from the 3 March 2014, 18:53 UTC
pass acquired at the University of Wisconsin to the VM, and place it in the correct directory for
display through a browser. To view the image in the Google Maps interface, wait a minute or two,
and then navigate your browser from any machine to the domain defined by your installation:
http://hostmachine.your.domain:8001/
Use the MODIS widget in the left set of navigation tools, to expand the directory tree and scroll
down to the “Aqua MODIS Short Wave Daytime Infrared Swaths” box, as shown below. To
display the image, click on the plus sign in the box, or drag and drop the box into the Google
Map region.
9
When the image has been uploaded, it will appear in both the single Swath display and the
Composite display. There are separate composites for daytime versus nighttime orbits for each
24 hour period. This information is determined based upon the sunrise and sunset times for the
latitude and longitude provided during the installation process. Once an image has been loaded
into the RealEarth Google Maps display area, the “Displayed” Tab under the “Layers” section
will be shown (see example below). Under this tab, all displayed products will be listed, and
users can then make each image product more or less transparent to the background (slider),
delete the image, or get information about the display. More than one image can be displayed
at a time, with the layering adjustable (drag and drop layers up and down in the table of
contents to rearrange their display order). See section 5 below for more information on the
display controls.
10
File uploads can be manually repeated for each of the files generated by the Polar2Grid software,
or the upload command can be added to an automated Polar2Grid process running at your site.
Wildcards can also be used in the file upload command. For example, imapp-upload *.tif,
will copy over all of the .tif files found in the current directory. If you attempt to upload many files at
one time, the uploads will automatically be queued to prevent saturation of the VM.
By default, seven days of imagery for each product will be kept on your VM. This is limited by disk
space availability (as assigned during setup). The number of days of imagery for each product can
be modified by using the “Product Manager” as described in section 4 below.
Section 4: Product Manager for IMAPP-RealEarth
4.1 Using your Product Manager
Once GeoTIFF imagery is uploaded to the IMAPP RealEarth VM, the resulting products can be
managed by using the “Product Manager” at
http://hostmachine.your.domain:8001/manager/
11
The initial password for the “wms” user (general product management) is “wm5”. There is also a
VM admin login “wmsadmin” (server administration) with password “wm5admin” found at
http://hostmachine.your.domain:8001/admin. We recommend changing both the
wms and wmsadmin web interface passwords as soon as possible after VM installation.
This should be done by running the following scripts on the Virtual Machine. Log into the VM in a
linux terminal, for example ssh –Y wms@<name of your VM>, or ssh –Y wmsadmin@<name of
your VM>, and execute these commands found in the VM /home/wms/bin directory):
set_web_password wms
and
set_web_password wmsadmin
Using a browser, logging into the manager as wms user will bring up the following interface:
The Manager allows IMAPP RealEarth administrators to manage and check the status of their
products. There are four basic components:
12
•
•
•
•
•
The header of the page (Top of the page to the right of the “SSEC RealEarth Manager”
title), it is always visible during scrolling, and identifies the current VM and reports the VM
CPU load averages (1min., 5min., and 15min.). Colored strip below the header indicates
total product status (green for “OK”, red for “late” – “late” refers to the expected product
frequency, such as daily, and would indicate that there may be a problem with an automated
upload script).
“Do not allow new products” and “Allow new products” buttons allow the administrator to
easily toggle the capability to upload new product types off and on. The button that is lighter
indicates which one is currently active.
“Advanced Configuration” lets VM administrators set basic configuration of the IMAPP
RealEarth user interface such as default product to display, center point and zoom level of
initial display and which user interface (UI) features to include. It also allows the user to
insert their own logo;
“Proxy Node List” allows linking to multiple local or remote VM nodes;
“Products List” initially lists placeholders for the 47 default IMAPP RealEarth products.
Once users upload imagery, this is where they can check its status (color coded green or
red), gain access to product details (inspect), modify individual product settings (edit), or
view a log file (log). Users can also check the general “upload log” for status of all product
upload activity.
Section 5: Using the IMAPP RealEarth Interface
This section describes the features of the IMAPP RealEarth user interface.
13
5.1 Basic Visualization
From your webpage at:
http://hostmachine.your.domain:8001/,
the Google Maps interface of IMAPP RealEarth allows common web-map functionality with several
enhancements. The interactive map allows users to pan and zoom their imagery, add multiple
images, arrange imagery layer order (drag-and-drop), adjust transparency, and turn overlay labels
on or off.
14
5.2 Composites and Animation
Imagery that is collected by polar orbiting satellites (such as Aqua, Terra, and Suomi NPP) in
multiple passes can be combined into a single layer. The determination of day versus night is done
based upon solar tables for the data date and the latitude and longitude specified during VM
installation. Time and Animation controls in the left panel let users run animation time-steps by
relative time (e.g. 2-hours ago from now); absolute time (e.g. 2013-10-08 16:09 UTC); or by product
(e.g. 4-days’ worth of ConUS True Color - Aqua). When multiple layers are displayed, users can
choose which product defines the animation parameters. Be sure to click “Set Times” for any
animation time or product changes to take effect.
5.3 Keyboard Shortcuts and Animation Control
Once an animation is loaded, users can use the animation controller to play the animation,
adjust its speed, step through a time-series manually, or change the mode between “loop”
(forward) or “rock” (back and forth). If the animation is paused, users can step through the
animation manually, forward and back with the left and right keyboard arrow keys ( ← and → ).
5.4 Sharing
There is a “Share” button at the top right of the “Display Controls” that allows users to share a
display with colleagues and friends or export it to Google Earth. The sharing function captures
the current display and animation settings to the users clipboard as a fully functional URL that
can be pasted in an email. The “KML” link generates a KML file for the current display that can
be saved locally and opened in Google Earth.
5.5 Settings
By clicking on the “Settings” button at the top left of the “Display Controls,” users can toggle
between their local time zone and UTC for time display at the top of the map.
15
Section 6: Additional Features
6.1 Uploading Other Files
The IMAPP RealEarth VM can be used to upload and display any GeoTIFF file even if it was not
created by Polar2Grid. Navigating to:
http://hostmachine.your.domain:8001/upload/
will bring you to an upload form:
Choose the file you wish to upload using the “Browse” button, fill in the Name, Date and Time
fields and click on the Upload button (Note: dashes are ok in a name but NOT underscores).
Use the RealEarth Manager to monitor the progress of the upload. If it is successful, a new
product field will appear in the Product Manager product list using the name you entered for
your file. It will not appear in the list of available products to display from your VM in the website
selector until you choose for it to be released by checking the “Released” box in the Product
Manager product “Edit” window.
16
Section 7: Trouble Shooting
This section provides suggestions to resolving problems you may have in the installation,
display or management of your IMAPP RealEarth VM. If these suggestions do not solve your
problem, please feel free to use the “Contact US” link at the bottom of the IMAPP website:
http://cimss.ssec.wisc.edu/imapp/
and include a detailed description of the problem, information about your host computer system,
and any error messages you might have received.
7.1 IMAPP RealEarth Installation
•
My initial IMAPP RealEarth KVM set up was not successful
Please verify that you system meets the requirements outlined in Section 1.2, and
contact your system administrator for help.
•
I executed the install_realearth-imapp.sh SSEC-RealEarthIMAPP.img installation command and it failed.
If you run into problems, please follow the instructions provided by the script. If you are
still unsuccessful, please make sure that your system meets all of the requirements
outlined in Section 1.2.
7.2 IMAPP RealEarth Products
•
I uploaded products, but they are not appearing on the VM
It may take a few minutes for the files to appear on the IMAPP RealEarth Virtual
Machine, depending on the size and number of files as well as internet speeds. If you
are uploading a number of files, please allow a couple of minutes for each file. The
upload script purposefully spaces-out the transfer of the files so that it will not overload
the Virtual Machine. If your file still does not appear, check the Product Manager log
files to see if it identifies a problem.
•
Uploading my own GeoTIFF file has failed
Any well formed GeoTIFF file should be useable. If it does not appear in the Product
Manager, check the “Upload log” for any messages concerning your file. If not, please
confirm that your GeoTIFF file is not corrupted (perhaps by testing it in desktop
software). If you are not seeing a product from a custom GeoTIFF, upload, check the
“Manager” to see that it is marked “released.”
7.3 IMAPP RealEarth Virtual Machine
•
My RealEarth VM is not responding (It will not display anything through my
browser)
17
If your RealEarth Virtual Machine is no longer responding, follow these commands as
user “root” to restart it.
sudo virsh shutdown [vmname]
Wait a minute or so to see if the VM shuts down cleanly.
sudo virsh list --all
If the “State” of your VM does not say “shut off”, force the VM to shutdown:
sudo virsh destroy
[vmname]
Then, restart the VM:
sudo virsh start [vmname]
18