NRM Air v2.2 User Guide
Chapter 5: Managing Monitoring Data
TABLE OF CONTENTS
Introduction ...................................................................................................................................... 5-33
NRM Workbench ....................................................................................................................................... 5-33
NRM Air Excel Add-In .............................................................................................................................. 5-33
Helper Access Database (AirWaterChemistry.mdb) ................................................................... 5-33
Geographic Extents................................................................................................................................... 5-44
FME & the NRM Workbench ........................................................................................................ 5-55
The Air Data Input Process.................................................................................................................... 5-55
Download & Prepare to Install the NRM Workbench ................................................................. 5-55
Installing the NRM Workbench (FS Users, Only) .......................................................................... 5-55
Initial Set-Up........................................................................................................................................... 5-88
Using the NRM Workbench ................................................................................................................ 5-109
1. Export Template Service ........................................................................................................ 5-1010
2. Validate Service ......................................................................................................................... 5-1313
3. Load Service ................................................................................................................................ 5-1514
FME Server Status ................................................................................................................................ 5-1716
NRM Air Excel 2010 Add-In ....................................................................................................5-1918
The Air Data Input Process............................................................................................................... 5-1918
Download & Install .............................................................................................................................. 5-2019
When to Install a New Version of the NRM Air Excel Add-In .............................................. 5-2423
Activating the NRM Air Excel Add-In ........................................................................................... 5-2625
Sample Definition Form ..................................................................................................................... 5-2827
Navigating Data in the Workbook.................................................................................................. 5-3130
Data Columns .................................................................................................................................... 5-3130
Display Range of Measurement ................................................................................................. 5-3130
Sorting Data ....................................................................................................................................... 5-3231
Copying & Pasting Data ................................................................................................................. 5-3332
Add-In Menu Groups ........................................................................................................................... 5-3332
Lab Data Group ................................................................................................................................. 5-3332
Analysis Group .................................................................................................................................. 5-3433
Server Group ..................................................................................................................................... 5-3433
Control Group ................................................................................................................................... 5-3534
NRM Air v2.2 User Guide
Managing Monitoring Data
Charting With the Excel Add-In ...................................................................................................... 5-3534
Run Chart Set(s) ............................................................................................................................... 5-3736
Chart Set XML Files ......................................................................................................................... 5-3736
Usage Scenarios ..........................................................................................................................5-3837
Scenario 1: Lab Usage & Maintenance ........................................................................................ 5-3837
Step1: Lab Data Samples.............................................................................................................. 5-3837
Step 2: Establish Definitions for Lab Data Samples .......................................................... 5-3938
Step 3: Establish Lab Data Measurements Standard Set ................................................. 5-4039
Step 4: Entering Measurement Data ....................................................................................... 5-4039
Step 5: Sending Lab Data to the Forest Service .................................................................. 5-4140
NRM Air Add-In Maintenance ..................................................................................................... 5-4140
Sending an Up-to-Date Air Water Chemistry Database to the Lab(s) ......................... 5-4140
Scenario 2: Loading From Excel Lab Data ................................................................................. 5-4241
Step 1: Receiving Lab Data .......................................................................................................... 5-4242
Step 2: Using Menu Groups ......................................................................................................... 5-4342
Step 3: Link Tables ......................................................................................................................... 5-4343
Step 4: Export an Air Water Chemistry Database .............................................................. 5-4443
Step 5: QA/QC & Simple Analysis ............................................................................................. 5-4646
Step 6: Use Tabular Menu Group .............................................................................................. 5-4646
Step 7: Using Charts Menu Group ............................................................................................ 5-4746
Step 8: Load from Excel................................................................................................................ 5-4847
Step 9: NRM Workbench, Validate, & Load .......................................................................... 5-4848
Scenario 3: Load from NPSTORET Back End ............................................................................ 5-5049
Step 1: Receiving a New NPSTORET Back End ................................................................... 5-5050
Step 2: Using Menu Groups ......................................................................................................... 5-5150
Step 3: Link Tables ......................................................................................................................... 5-5151
Step 4: Export an Air Water Chemistry Database .............................................................. 5-5251
Step 5: Load from NPSTORET.................................................................................................... 5-5251
Step 6: QA/QC & Simple Analysis ............................................................................................. 5-5554
Step 7: NRM Workbench – Validate & Load ......................................................................... 5-5655
Scenario 4: Analyze Historical Lab Data..................................................................................... 5-5756
Step 1: Export an Air Water Chemistry Database .............................................................. 5-5857
Step 2: Using Excel NRM Air Add-In with Historical Data .............................................. 5-6159
Step 3: Menu Groups ..................................................................................................................... 5-6160
Step 4: Link Tables ......................................................................................................................... 5-6261
Step 5: Retrieve Data from the Air Water Chemistry Database ................................... 5-6261
Step 6: QA/QC & Simple Analysis ............................................................................................. 5-6361
Additional Information ............................................................................................................5-6463
Post-Installation (Non-Lab) ............................................................................................................. 5-6463
Access and User Interface ................................................................................................................. 5-6463
Office Excel 2010 File Content ........................................................................................................ 5-6463
Excel & Custom Content .................................................................................................................... 5-6463
5-2
NRM Air v2.2 User Guide
Managing Monitoring Data
INTRODUCTION
The Air program made architectural decisions to create tools to: 1) use Excel to host
application software as a Microsoft Office add-in called the “NRM Air Excel Add-In,” and 2)
use the Feature Manipulation Engine (FME) Server and to create a data loader tool, named
“NRM Workbench.” This solution creates an efficient method of working with water
chemistry lab data, outside of the NRM Air centralized database environment. These tools
are used for data entry, QA/QC, simple analysis and extracting data from the NRM Air
database prior to loading new data into the database. After the new data QA/QC is
complete these tools can then be used to load the verified data into the NRM Air database.
There are two applications needed to operate these tools:
1. The NRM Workbench, and
2. Microsoft Excel 2010 (or higher) to run the NRM Air Excel Add-In
In addition to these applications, an understanding of how to work within the “geographic
areas or extents” associated with the Air Program and the NRM Air centralized database
will be necessary. These tools will be addressed later in this guide.
NRM Workbench
The NRM Workbench is the user interface developed for importing and exporting data into
and out of the NRM Air centralized database system. There are three functions within the
NRM Workbench for exporting and loading air water chemistry lab data:
1. Export Template
2. Validate
3. Load
NRM Air Excel Add-In
The NRM Air Excel Add-In allows for consistency in how lab data is loaded into the NRM
Air database. This Add-In also has the capability for charting and simple analysis of data.
The NRM Air Excel Add-In provides:
•
•
Menus, forms, charting, and Microsoft Excel data manipulation
Data transformation into and out of a Microsoft Access database
Helper Access Database
The “helper” database is a “behind the scenes” tool that does not contain data, but assists
the NRM Air Excel Add-In in manipulating data while loading into the
AirWaterChemistry.mdb database. The helper database is also involved in loading data
from NPSTORET into the Air database.
Users will not access the helper database directly unless a new helper database needs to be
generated, or an existing helper database has changed folders and needs to be located. The
5-3
NRM Air v2.2 User Guide
Managing Monitoring Data
helper database file takes on the name of the Excel file open, with “helper.accdb” added to
the file name.
Geographic Extents
While most FS work is separated into “regions,” the NRM Air centralized database is
separated into five geographic “extents.” The map below defines the NRM Air Geographic
Extents. It is recommended to set the default geographic extent in the initial set-up of the
Excel Add-In to the area used most frequently. The extent can be changed at any time, in
Excel, within the Lab Data box when the add-in is active.
5-4
NRM Air v2.2 User Guide
Managing Monitoring Data
FME & THE NRM WORKBENCH
The Air Data Input Process
While some of the details necessary to use the NRM Workbench and Excel Add-In are
included in this section, please see the “Usage Scenarios” section at the end of this Chapter
for specific examples of data flow.
After water chemistry lab data is received by the AIR team from various lab sources, it
should be reviewed using the NRM Air Excel Add-In to ensure data integrity. Then it is
ready to be loaded into the Air database for centralized storage, review and further
analysis. To load this data Air uses the FME Server and the NRM Workbench application.
The FME Server is a queued environment where requests for processing are performed in a
first-in-first-out (FIFO) priority. Requests may be sent or “invoked” synchronously or
asynchronously. Requests done using the synchronous method may encounter long wait
times due to other items queued on the FME Server. The preferred (and default) method is
to invoke the service asynchronously, receiving an email notification from
fmeadmin@fs.fed.us when the tasks are completed.
Download & Prepare to Install the NRM Workbench
The NRM Workbench application is maintained in Team Forge and a version history can be
reviewed here: http://sforge.fs.fed.us/sf/go/doc21261. The Forest Service Tivoli
distribution should take care of all prerequisites with the exception of the I-Web Client,
however you will want to make sure that you have:
1.
2.
3.
4.
Administrative privileges – http://gadgets.ds.fs.fed.us/EUTools/ad_promote.asp
Microsoft .NET Framework 4
Oracle 11g Client – http://tod/
I-Web Client – http://iweb.fs.usda.gov/gisutils/iwebclient
Installing the NRM Workbench (FS Users, Only)
Before you install the NRM Workbench, you must make sure you have admin privileges. To
do this, go to the Forest Service “Gadgets” website
(http://gadgets.ds.fs.fed.us/EUTools/ad_promote.asp) and promote yourself so you have
administrator privileges.
If you have previously installed the NRM Workbench, please uninstall it. Use the “Add or
Remove Programs” control panel to look for NRM Workbench and click Remove.
Download the latest version of the NRM Workbook (data loader tool) from
http://sforge.fs.fed.us/sf/docman/do/downloadDocument/projects.nris_water_uses_tracking/doc
man.root.outputs.fme/doc21261. You will be asked whether to install this program or save it
to your hard drive. Click on Save to save the program to your hard drive. The directions
below apply whether you are installing the Workbench for the first time, or installing
an update to the Workbench.
5-5
NRM Air v2.2 User Guide
Managing Monitoring Data
Find the installation program file and double click on it to open the install wizard. The
following steps walk you through setting up the NRM Workbench on your computer.
1. To install the NRM Workbench data loader, double-click on the install program icon.
2. Click on Next.
5-6
NRM Air v2.2 User Guide
Managing Monitoring Data
3. Browse to where you want to save the Data Loader tool, then click on Next. We
recommend leaving the default path for ease of support. Do not install this in a
user’s folder.
4. Click on Next to install the program.
5. You will see a progress bar while the Data Loader installs.
5-7
NRM Air v2.2 User Guide
Managing Monitoring Data
6. You will receive a notice when the installation is complete. Click on Close to exit the
install wizard.
This installs a shortcut link, labeled “NRM Workbench”, on your desktop. This can be
moved to any location you find more convenient.
Initial Set-Up
The very first time you run the NRM Workbench, you will have to enter an email address.
This email address will be passed to the FME Server and used for asynchronous requests to
send you notification when the requested process is complete.
In addition to providing your email address, options for selecting the database and FME
Server environments are available. Leave the “Production” option selected for both
environments and click OK.
5-8
NRM Air v2.2 User Guide
Managing Monitoring Data
The NRM Workbench is used for other business area applications. To use the AIR services,
use the Config->Select Services menu item.
Click the display options to “true” for the AIR Services you intend to make use of. These
options can be changed at any time.
Click on OK in the lower left-hand portion of this window to save your configuration. This
completes the initial setup of the NRM Workbench.
If you are updating an existing installation with a new version of the Workbench, you will
want to make sure that you select Config -> Reset before selecting services.
5-9
NRM Air v2.2 User Guide
Managing Monitoring Data
Using the NRM Workbench
The NRM Workbench user interface is simple. From the initial page, use the drop-down
arrow to “Select the Workbench Service.” The services available are the ones chosen in the
initial NRM Workbench setup:
Click on the desired service (most processes will begin with the EXPORT TEMPLATE task),
fill in the required information for that service, then click the “Invoke the Service” button.
The application also provides “Description” information for each service selected. To use
this option, first select the service, then click on the “Description” button. A new window
will open providing a description of that particular service:
Please refer to the Scenarios section at the end of this Chapter for detailed examples of how
the NRM Workbench and NRM Air Excel Add-In are used to manage data.
1. Export Template Service
There are two reasons to make use of the Export Template service: 1) to get an up-to-date
AirWaterChemistry.mdb Access database (template) for population of lab data to load into
the centralized NRM Air database, or 2) to get an export of a specific project(s) water
chemistry lab data to use in the NRM Air Excel Add-In for review and simple analysis.
Exporting lab data related to projects, sites, or HUC are the only situations when the
geographic extent must be included.
5-10
NRM Air v2.2 User Guide
Managing Monitoring Data
To acquire an up-to-date AirWaterChemistry.mdb Access database (template) to use for
populating and importing new lab data into the NRM Air database, leave all fields blank.
In some cases it is only necessary to acquire data from a single or small group of
monitoring projects, for simple analysis and review. To select monitoring projects, doubleclick in the text field to bring up the selection window. The Monitoring Projects listed will
only be those related to the geographic extent you selected. One or multiple projects can be
selected from the list presented. Select OK when complete.
The “Delivery Method” identifies whether the request sent to the FME Server should be
submitted and the response delivered synchronously or asynchronously. Remember:
requests done using the synchronous method may encounter long wait times due to other
items queued on the FME Server. Because the FME Server is a FIFO-based system, we
recommend that all service requests are submitted asynchronously. This is the default
setting.
Click the “Invoke Service” button in the upper right corner of the window to send the
workbench service request to the FME Server. The request is sent as an HTTP request to
the FME Server and its response is displayed below the request section of the window.
5-11
NRM Air v2.2 User Guide
Managing Monitoring Data
If the request had been sent synchronously, a successful response would include a link to
the output of the request. The output is dependent on which request was invoked. The
output of the Export Template service is an Access database, named
AirWaterChemistry.mdb.
Email notification from fmeadmin@fs.fed.us will be received for asynchronous requests.
Depending on the current usage of the FME Server and the Forest Service email network
systems the amount of time it takes to get the email may vary. The email you receive
should look similar to this:
NOTE: Because the asynchronous email response lacks any information that relates it to
the request without downloading and opening the output, it is recommended only one
workbench service request be invoked at a time and its output dealt with before invoking
the next request.
5-12
NRM Air v2.2 User Guide
Managing Monitoring Data
After receiving the email from fmeadmin@fs.fed.us, click on the download link to open the
zip file containing the results of the request. A prompt to either open the file or save it to
your hard drive will appear.
The content of the “Export Template” zip file is the AirWaterChemistry.mdb file. Open
and/or save the AirWaterChemistry.mdb file in a location where it can be easily accessed
when needed for use with the NRM Air Excel Add-In.
2. Validate Service
The Validate service is used to send a request to the FME Server and have it process a
newly populated Air Water Chemistry Access database template without performing the
actual inserts and/or updates to the NRM Air database. It performs the same procedure as
the Load service, with exception of the inserts and/or updates. Optionally, the reporting of
warnings can be turned on or off. For example, most warning messages will appear when
there are no child records related to a parent record, or no sample records for a visit.
5-13
NRM Air v2.2 User Guide
Managing Monitoring Data
By default, the Report Warnings in Results option is set to “No” in an effort to reduce the
output size. Depending on the number of records being processed, reported warnings may
produce a large verbose output and may not add any value. If you wish to have warnings
included in your final report, click “Yes.”
As with the Export Template service, data can be received either synchronously or
asynchronously.
When you have made the correct selections, click on the “Invoke Service” button to run the
process. FME will provide a notice to the NRM Workbench Runner window as well as
sending you an email with a link to download the results.
The output from this service is a single HTML file contained within a zipped file you can
download and save to your computer. You should always review the report for errors and
completeness before invoking the Load service. Also notice that in the report’s first row of
the Validate and Load workbench services the database environments and lab data source
are identified. If the exported Access database environment (Production, Test, or
Development) doesn’t match the currently selected database environment, no further
processing occurs.
5-14
NRM Air v2.2 User Guide
Managing Monitoring Data
NRM Air Water Chemistry Lab Data, Validate Report
Updates
Messages
Database
Environments
match and lab
data source is
known.
Record Type
Supplied Information
Inserts
Old
Values
New
Values
Export Database Environment =
Production
Export Database = IDB
Export Schema = FS_NRIS_AIR
Active Database Environment =
Production
Active Database = IDB
Active Schema = FS_NRIS_AIR
Lab Data Source = EXCEL
The Validate service can be run as many times as needed. For instance, you may discover
data are missing and you will need to make corrections. Make the corrections in the Excel
Workbook and perform the “Load from Excel” procedure to repopulate the Water
Chemistry Access database before initiating the Validate service again. You will need to
reselect the updated Access database before invoking the service. Make sure to review the
report from the Validate service each time to ensure it identifies correctly what will take
place when the Load Service is used.
After successfully using the Load service, running the Validate service again (post-Load)
should report that nothing needs to be inserted or updated. This isn’t required, but might
be a good check to perform.
3. Load Service
The Load service is used to send a request to the FME Server and have it process a newly
populated Air Water Chemistry Access database, and perform the actual inserts and/or
updates to the NRM Air database. The Load service is identical in steps as the Validate
service and provides a similar HTML report, identifying what records were inserted and
updated as well as identifying existing records not requiring any updates.
Before invoking the Load service, review the Validate report and be 100% certain it
contains what you expect. First, discover if there are any reported errors. For readability,
you can view the report in a browser like Internet Explorer or open in it Microsoft Word.
Perform a find with the word: “error” and advance to the locations if any errors are
present.
5-15
NRM Air v2.2 User Guide
Managing Monitoring Data
After checking for errors in the Validate report, you should also review it to understand
what exactly what will occur in the load. Thinking critically, ask yourself:
•
•
•
Are the records to be inserted new?
Do the records being updated exist?
Are the values of the records being updated are correct?
If Sites, Visits, and/or Samples have been created in the Air application prior
to loading the data, make sure any data changes are the right ones to make.
NOTE:
There are no roll-back operations implemented in this process. When the load is run, each
step is independently taken and is either successful or not. If errors occur and some data
doesn’t get loaded, manual corrections to the populated Air Water Chemistry Access database
may be necessary or a fix to upstream processes may be needed and re-run. If data is
accidentally loaded to the wrong project, site, or visit, it will require a manual deletion of the
data in the NRM Air application or request to Support to run an SQL process to remove the
data in question.
The Load service can be run more than once to make corrections as well. Use the combination
of Validate and Load service requests to ensure the outcome is what is desired.
ADDITIONAL NOTE:
For all data which receives a “Valid” response in the Validity test described earlier, the data is
automatically “published” in the NRM Air database. This means that this data is immediately
available in the Air application and, on the next cycle, will be automatically sent to the FED
website for use in analysis (see Chapter 9 for more on the FED website).
All data receiving either an “Invalid” or “Unclear” response in the Validity test is noted as “not
published” in the NRM Air database. This means that the data is immediately available in the
Air application, but it will not be sent to the FED database until new information is added or
data is rerun and a “Valid” response is received.
If your data will be published to the FED website, please note that this update occurs on an
automatic weekly cycle, so look on the FED website in the first part of the week following your
data upload to Air.
Should the data you load receive a “Valid” response but you do not want it uploaded into the
FED database, don’t worry, that is easily fixed. As soon as you upload the data, confirm that
the information is available, and on the Monitoring Visit form, and uncheck the Publish? box.
5-16
NRM Air v2.2 User Guide
Managing Monitoring Data
FME Server Status
The status of the jobs submitted to the FME Server can be viewed while they are being run,
in the queue to be run, or have been completed. To view the jobs currently running, use the
Help menu, and select FME Server Status. The FME REST Service response is displayed in
the lower region of the window.
Follow the running link to see the jobs currently executing on the FME Server. For each
request being executed, a Job ID and the status information about the job are listed. If no
jobs are currently executing, then no jobs are identified.
The Internet explorer capabilities, Back and Refresh, are implemented in the right-click
menu of the FME response region of the window.
5-17
NRM Air v2.2 User Guide
Managing Monitoring Data
5-18
NRM Air v2.2 User Guide
Managing Monitoring Data
NRM AIR EXCEL 2010 ADD-IN
This Add-In will work in any Excel 2010 workbook. The interface is implemented on a new
Tab within the Excel menu system. The Excel 2010 Water Chemistry Lab Data workbook
Add-In has dependency on the .NET Framework and Microsoft Office. The version of these
platforms for which this application has been tested is:
•
•
.NET Framework 4.0
Microsoft Office 2010 Professional SP1
Both Microsoft Excel and Access programs are needed. If another version of Microsoft
Office is installed that doesn’t include Access, you will need to install Microsoft Access
2010.
There may be occurrences of the Microsoft Office installation that does not include an
additional runtime needed for the Add-In. This runtime is called “Visual Studio 2010 Tools
for Office Runtime” and can be downloaded from Microsoft, at:
http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=20479.
The Air Data Input Process
While the details of the processes needed to use the NRM Workbench and Excel Add-In are
included in this section, please see the “Usage Scenarios” section at the end of this Chapter
for specific examples of data flow.
After water chemistry lab data is received by the AIR team from various lab sources, it
should be reviewed using the NRM Air Excel Add-In to ensure data integrity. After that, it is
ready to be loaded into the Air database for centralized storage, review and further
analysis. To load this data Air uses the FME Server and the NRM Workbench application.
5-19
NRM Air v2.2 User Guide
Managing Monitoring Data
The FME Server is a queued environment where requests for processing are performed in a
FIFO (first-in-first-out) priority. Requests may be sent synchronously but you may
encounter long wait times due to other items queued on the FME Server. The preferred
(and default) method is to receive email notification when the tasks are completed,
invoking the services asynchronously.
All original/“native” Units of Measure (UOMs) and values are what should be provided to
the Workbench for Loading into the Air database.
Download & Install
Before you install the Excel Add-In, you need to make sure that you have “admin privileges”
activated (using the USDA Forest Service End User Gadgets page:
http://gadgets.ds.fs.fed.us/EUTools/ad_promote.asp). You will also need to make sure that
you have Excel 2010 and Access 2010 installed on your machine; you can get these from
the Tivoli On Demand site (http://tod).
The NRM Air Excel Add-In is maintained in Team Forge and a version history can be reviewed
here: http://sforge.fs.fed.us/sf/go/doc22294. Please coordinate delivery of this externally
(to non-FS users) via Ann Mebane or Sharon Grant.
The following steps walk you through installing the NRM Air Excel Add-In functionality on
your computer.
1.
Go to the FS Gadgets page (http://gadgets.ds.fs.fed.us/EUTools/ad_promote.asp) to request
“admin privileges” to allow installation of the Excel Add-In.
2. Exit out of all Excel applications, if any are open.
3. Download the latest version of the Add-In installation package
(http://sforge.fs.fed.us/sf/docman/do/downloadDocument/projects.nra/docman.root.data
_migration.lab_data.workbook/doc22294); save the file to your desktop.
4. Exit out of all Excel applications, if any are open.
5. To install the Excel Add-In, double-click on the install program icon.
5-20
NRM Air v2.2 User Guide
Managing Monitoring Data
6. Click on Next.
7. Browse to where you want to save the Excel Add-In, then click on Next. It is recommended
to use the default location. Do not install the NRM Air Excel Add-In in a user folder.
5-21
NRM Air v2.2 User Guide
Managing Monitoring Data
8. Click on Next to install the program.
9. You will see a progress bar while the Add-In installs.
5-22
NRM Air v2.2 User Guide
Managing Monitoring Data
10. As part of the installation, you are asked to select a default geographic extent for your work.
This does not mean this is the only extent you can work in, just the default extent that will
open when you run the NRM Excel Add-In.
11. A notice will appear when the installation is complete. Click on Close to exit the install
wizard.
Should you receive a dialog box stating that the “Publisher cannot be verified,” click on Install to
continue the NRM Excel Add-In installation.
5-23
NRM Air v2.2 User Guide
Managing Monitoring Data
If, when you open Excel for the first time after installing the Add-In and you do not see the
NRM Air tab, go to the File tab and select Options. This opens the Excel Options window.
Click on the Add-In button in the left column. At the bottom of the Add-In window, click on
the drop-down arrow next to the Manage field and select Disabled Items. Then click Go.
The Disabled Items window opens. Click on the item you would like to enable, then click on
Enable. You are then brought back to the Excel Options window; click OK to exit.
When to Install a New Version of the NRM Air Excel Add-In
When a new version of the Excel Add-In becomes available, the users and labs will be
notified by Ann Mebane or Sharon Grant so that everyone is working from the latest tools
and there won’t be any issues with compatibility of the spreadsheets.
5-24
NRM Air v2.2 User Guide
Managing Monitoring Data
The major and minor version (1st and 2nd, in this case 2.2) numbers MUST MATCH the NRM
Air application version. Do not use a mis-matched version. The third number represents
the version of the Add-In, which should match the TeamForge document version (in this
case 27.)
When an Excel workbook is enabled with the Add-In, the major and minor version
numbers are embedded in the workbook file. If a newer version of the Add-In is used with
an older workbook file, unexpected results may occur and the following message should
appear:
For cases like this, it would be best to create a new workbook with the new version of the
Add-In, and then cut/paste (the values) in the Lab Data worksheet from the old version
workbook into the new.
It’s difficult to project what might change in future versions, but here are a couple forward
thinking considerations.
When copying the data, make sure you are copying into the correct columns. Additional
columns may have been added (or some removed) in the new version. It may also be wise
to use the Sample Definition form on each row to verify the LOV values are still valid.
Instead of copying the measurement header rows from the old workbook, it would be best
to recreate them with the new version. Add the measurement headers prior to copying the
data into the worksheet.
5-25
NRM Air v2.2 User Guide
Managing Monitoring Data
Activating the NRM Air Excel Add-In
Each time you access Excel and want to use the NRM Air Add-In, you will need to activate
the Add-In. Please be aware that, if you have not already done so, you will be prompted to
save your file before activating the Add-In.
While the Add-In is enabled, all Excel workbook files opened within the same instance of
Excel will be enabled. If you are finished working with lab data but still wish to use Excel,
remember to uncheck the Water Chemistry Lab Data box to turn the NRM Air Add-In off
before opening another Excel File. Alternatively, you can just close the Excel Application
and open another one to turn off the add-in.
To work independently on an Excel file that you don’t intend to use with the NRM Air AddIn turned on, starting another instance of the Excel application, via the start menu, can be
done concurrently. Excel is a multi-document application; double-clicking on an Excel file
will make use of the already opened instance of Excel.
5-26
NRM Air v2.2 User Guide
Managing Monitoring Data
Enabling the LabData workbook sets up the necessary functionality for the location where
the workbook is being used. A check is made on the computers fully qualified domain
name (FQDN) and if the computer is not a USDA Forest Service system (domain other than
“fs.fed.us”), then it has limited functionality. Only the Lab Data menu group is added to the
NRM Air tab in this case. When the system is a Forest Service computer, the Lab Data,
Analysis, and Server menu groups are added to the NRM Air tab.
There are four additional actions that occur when the LabData Workbook is enabled (for
the first time) with each Excel document:
1. A helper Access database is copied to the location where the workbook is saved.
The name of this file is created using the name of the Excel document, appending
“Helper.accdb” at the end of the file name.
2. A custom XML part is added to the Excel document for keeping track of variables
specific to each workbook.
3. A worksheet is added to the workbook, named LabData, and some formatting occurs
within the worksheet.
4. Tables in the helper database are linked to the AirWaterChemistry.mdb database
located in the installed folder.
Some of the processing in the Add-In takes an extended amount of time to complete,
depending on the amount of data. The Add-In is designed to perform these procedures
while displaying a popup dialog with a progress bar. At the end of these processes, you will
be notified when it is complete.
5-27
NRM Air v2.2 User Guide
Managing Monitoring Data
Should you encounter any processes while using the Excel Add-In that take some time to
perform or to populate the workbook with sample and measurement data, you will see a
progress bar to let you know the application is working.
Sample Definition Form
The Sample Definition form is used by the Labs for data entry. A row is created in the
spreadsheet that contains all of the information entered into the form. This form is
accessed from the Lab Data group by clicking on the arrow for Samples and selecting either
Water or Deposition. You will see the same form whether you choose “New …” or “Edit …,”
however when you select “Edit …” the form will be filled in with information based on the
cell you are on in the spreadsheet.
All fields with titles in bold and underlined are required fields which must be filled in prior
to clicking on “OK” and saving the sample definition. All fields with titles in bold (with no
underline) must be completed in order to save a sample. There are a variety of ways to get
information into some of the fields. There are some fields that can be typed into or have
data pasted into them:
There are some fields that include an arrow you can click for a drop-down list of values
(LOV):
5-28
NRM Air v2.2 User Guide
Managing Monitoring Data
There are date fields that include an arrow so you can display a calendar to select the date
from:
And, finally, there are fields which have a “Select” button next to them that brings up a
more extensive and descriptive LOV to choose from:
The Record Selection Dialog form provides the ability to quickly find the record you’re
looking for. In the “Scroll To:” field, type in the name of the record; for example, Diamond
Lake:
5-29
NRM Air v2.2 User Guide
Managing Monitoring Data
As you type in the name, the list automatically scrolls to match your characters. If we want
to automatically find Diamond View Lake, the “Scroll To:” field needs to be more complete:
After you have found the correct record, click on the name of the record to highlight it and
then click “OK” to place this record in the form:
5-30
NRM Air v2.2 User Guide
Managing Monitoring Data
If, at any time while editing the Sample Definition form, you want to clear out all of your
changes, simply click on the “reset form” button in the lower right corner of the form.
Navigating Data in the Workbook
Data Columns
All data columns have a format defined. Columns with numeric data are limited to five
digits to the right of the decimal place. It is important that you not change the formatting of
the data. Take extra precaution to only copy values when cut and pasting data into the Lab
Data worksheet.
Should you be working with a sample that has both a deposition and a water sample,
define the deposition sample first. The way the workbook collects the information that
goes into a visit record, the first sample row that identifies a new visit, that is the sample
the visit information is populated with and the Deposition sample has more information
that goes into a visit than the water sample.
Currently, the deposition sample row in the workbook populates the visit precipication
amount and collection method which the water sample does not. Alternatively the
deposition and water samples can be loaded with separate workbooks.
Display Range of Measurement
There may be times when you need to look at the range for a given measurement. You can
access the Customize Status Bar functionality by right-clicking on the gray bar at the
bottom of the Excel window below the Worksheet tabs.
5-31
NRM Air v2.2 User Guide
Managing Monitoring Data
You will want to use the Measurements/Hide Show Detail option in the Lab Data menu
group of the NRM Air Add-In tab to hide the limit values so they do not impact the Excel
column summarized values.
Sorting Data
You may have noticed that the LabData worksheet has quite a few header rows. If you
attempt to use traditional Excel column sorting methods, you will not get the results you’re
looking for. Follow these steps to sort data in the LabData worksheet:
1. Use the Lab Data group menu Sample -> Select All
2. Click the Excel Data Tab, then click on the Sort Table Icon (DO NOT use the A-Z or
Z-A options)
3. Leave the “My data has headers” box unchecked and use the “Sort By” in the column
labeled “Column” to specify which column in the worksheet you want to sort by
a. Should you wish to sort by multiple columns, use the Add Level function to
do so
b. All other options are available for you to specify what to sort on, the order,
whether to sort case-sensitive, and top-to-bottom or reverse
5-32
NRM Air v2.2 User Guide
Managing Monitoring Data
Copying & Pasting Data
Should you need to copy and paste data in the workbook, there are a couple of things you
need to keep in mind:
1. Make sure you paste only the values you have copied; cell formatting is relied on
for the Add-In to properly function.
2. To re-apply the formatting of the cells in the workbook, click on the Format button
in the Lab Data group and select the Re-Apply Formatting option
Add-In Menu Groups
Within the NRM Air tab, features of the Add-In are presented in menu groups. These
groups are labeled: Lab Data, Analysis, Server, and Control.
Lab Data Group
The Lab Data group provides the functionality of working with the LabData worksheet in
the workbook, setting of the regional extent, adding/editing sample information, and
adding/editing measurement parameters. You are not restricted from editing the
information in the worksheet directly, but in order to get valid values for many of the fields,
it’s best to create/edit the Sample data and the Measurement headers in the data entry
dialogs. The actual measurement values will need to be entered directly in the Excel
LabData worksheet.
5-33
NRM Air v2.2 User Guide
Managing Monitoring Data
Use the Measurements ->Add Standard Set to add the measurement headers for the
majority of the measurements the Forest Service are interested in. Do this BEFORE adding
additional measurement parameters. The standard set must start immediately after the
Sample data columns.
Make sure to use the original or “native” Units of Measure (UOMs) for all data when loading
into the Air database.
Analysis Group
This group contains tabular and charting analysis functions for examining the lab data.
• Tabular
o Group into Standard Set …
Used after retrieving data from NPSTORET or from an export from the NRM
Air database. This function will merge like measurements, ignoring the
equipment type, and the detection and quantification limits. This is not to be
used if the data has been entered into the Lab Data worksheet using the
standard set (Measurements->Add Standard Set)
o Add Cell Formulas (Standard Set Only)
Whether or not the Group into Standard Set has been used or the standard
set has been added with the Measurements menu, as long as the columns of
the standard set are where they need to be, this will add cell calculations
where values are not provided in the micro-equivalent and calculation only
columns.
o Compare Replicates
This function makes a copy of the LabData worksheet, sorts the data, and
attempts to find samples that have regular and duplicates, replicates,
removing samples that don’t. Then adds two additional rows and calculates
the average and the relative standard deviation of the measurement values.
• Charts
o Chart Sets
Provides the capability to create and run predefined sets of charts. Chart
types of Histogram, Scatter, and Box-Whisker are available. Chart Sets are
saved in the workbook file when the Excel document is saved. They can be
exported, then imported into other workbooks from the Chart Sets menu.
o Time Series Plots (with selected columns)
For each measurement column that has a cell selected, will create a scatter
chart worksheet with the Sample Collected Date as the X Axis.
o Delete all Chart Worksheets
This function removes the worksheets that have charts on them. Worksheets
that may have failed to generate a chart will need to be deleted manually.
Server Group
This group contains the functionality for working with the Air Water Chemistry Access
database (template). The following are the capabilities of this group:
5-34
NRM Air v2.2 User Guide
•
•
•
Managing Monitoring Data
Link Tables – creates the linked tables necessary from the helper database to the Air
Water Chemistry Access database to populate and/or read from.
Load – loads the Air Water Chemistry Access database with data from either an
NPSTORET database or from the Excel LabData worksheet.
Retrieve – reads data from the Air Water Chemistry Access database into the
LabData worksheet.
Control Group
The Control Group provides three functions:
1. Present an “about information” dialog to view the version and a short description of
the Add-In.
2. Put a check in the Water Chemistry Lab Data checkbox to enable the Add-In and
allow the Excel document to be used as a LabData workbook.
3. Provide the ability to locate the helper database if the workbook gets renamed or
moved to a different folder or re-generate a new one.
Charting With the Excel Add-In
There are some basic charting functions built into the Add-In. To access this, click on the
Charts button in the Analysis Group.
Before you begin any charting activities, be sure that you have narrowed down your data to
one location (particularly when working with Time Series Plots).
To begin creating charts, click on the Charts button in the Analysis Group of the Add-In.
Then, click on the Chart Set option, then select Configure …. This opens the Chart Set
Configuration window.
5-35
NRM Air v2.2 User Guide
Managing Monitoring Data
The Chart Set Configuration window is used to set up the different types of charts that you
will use frequently. To set up a new Standard Chart Set, click on the New button. You will
be prompted to provide a name for this new chart set.
Enter a descriptive name for the chart, then click OK. To assign specifics to this chart,
highlight the name in the list, then complete the information on the right side of the
window. When you have completed all of your edits, click on Save.
To delete a standard chart from the Chart Set Configuration list, highlight the name of the
chart and then click on the Delete button.
Charts you can create include box whisker, histogram, and scatter plots. The standard
chart set includes: NO3(BV), SO4(BX), Conductivity(DA, BM), and Cat-An(CR, CO). Box
whisker charts are frequently used in statistics; they identify such information as smallest
observation, lower quartile, median, higher quartile, and largest observation, as well as
outliers and averages. Histogram charts are a visual representation of the distribution of
the data selected. Scatter plots lets you see all of the individual points of data so that you
can identify clustering, linear patterns, etc.
5-36
NRM Air v2.2 User Guide
Managing Monitoring Data
Run Chart Set(s)
To run a chart set, go to the Analysis menu group and click on Charts, then Chart Sets.
Now select Run… to open a dialog box where you select which previously-configured chart
set you wish to use. Click the box next to the Chart Set name to run that chart set. Multiple
chart sets may be selected, however this will increase the amount of time needed to
process the data and create the charts.
When you click on OK, the Add-In processes all of the data based on the chart set(s)
selected. A new worksheet is created for each of the charts. When the process has
completed and all selected charts have been generated, you will receive a confirmation box.
Click OK to close this window and review the charts.
Chart worksheets are not created when the measurement parameter column(s) cannot be
found and if the column(s) of a chart are empty, the worksheet is added, but no chart is
created.
Chart Set XML Files
Chart sets can be exported and shared with other users of the Add-In or imported into your
own workbook files. When exporting, you can choose which ChartSets are exported, but
please note that the ChartSet import procedure overwrites the workbook’s definition of it’s
ChartSets, so you may want to export all ChartSets to ensure any workbook that imports it
will have the Standard ChartSet. The ChartSet XML files may be located anywhere you
want to save them. They must have an .xml file extension.
5-37
NRM Air v2.2 User Guide
Managing Monitoring Data
USAGE SCENARIOS
This Add-In will work in any Excel 2010 workbook. The interface is implemented on a new
Tab within the Excel menu system. The installation makes the Add-In available by default,
but you can turn off the Add-In via the Excel Options/Add-Ins management.
Scenario 1: Lab Usage & Maintenance
This scenario presents the steps the labs will take to use the Add-In for data entry and to
support the labs, keeping them up to date.
Step1: Lab Data Samples
The first thing that you’ll want to do is save a new Excel workbook file onto your hard
drive. To do this:
1. Open Microsoft Excel
2. Save the Excel file. Naming conventions are “yyymmddProjectNameR#.xlsx”
3. Enable the NRM Air Add-In
Be aware that while the Add-In is enabled, *all* workbooks opened within the same
Excel application will be enabled. This means that an Access database will be copied
to the same location as the Excel file, named with the Excel filename as a prefix to
5-38
NRM Air v2.2 User Guide
Managing Monitoring Data
“Helper.accdb”; a LabData worksheet will be added to your workbook; and a custom
XML file is added to the workbook to manage settings (such as the Extent). You can
open additional instances of the Excel application and the Add-In will not be active,
unless you activate it.
4. Under the Lab Data menu group, set the Extent and resave your file.
The extent influences the choices presented in such data as Lists of Values, Projects,
and Sites.
5. The Control menu group can help you locate where the database resides and to
generate a new helper database.
Should you move the Excel file and helper database, or lose the helper database, you
can use the Helper Database menu choices (Locate and Generate) to either select the
new location where the database resides (Locate) or to create a new helper
database (Generate).
Step 2: Establish Definitions for Lab Data Samples
Samples and all related higher-level information is entered via a Sample Definition form
and added into the LabData worksheet as a Row. Depending on whether the sample is a
surface water or deposition sample, certain fields will be identified as required. The fields
that are required before leaving the form are bolded and underlined. The fields that are
required to complete a sample are bolded.
5-39
NRM Air v2.2 User Guide
Managing Monitoring Data
Use the Samples (Water or Deposition)-> New or Edit menu choices to display the
Sample Definition form. To edit an existing Sample, select a cell in the Excel row before
using the Edit menu selection. The Sample Definition form is used to ensure data entered
is valid per lists of values, date, and time formats.
Once saved in the Excel row, the data can be manipulated normally. However, if you
change a value that is selected from a list, it is best to use the Sample Definition form to
reduce typing errors.
Step 3: Establish Lab Data Measurements Standard Set
Use the Measurements -> Add Standard Set menu choice to add the standard set of
measurements your spreadsheet. This is a set of measurements that are used most often.
This adds column headings beyond the sample headings. DO NOT delete these columns;
however, you can easily hide these columns, if desired.
After adding the standard set of measurements, use the Measurements -> New or Edit
menu choices to create additional measurement columns. To edit an existing Measurement
column heading, select a cell in that column before using the Edit menu selection.
You can use the Edit Definition form to set the Equipment Type, Detection, and
Quantification limits for the standard set of measurements that you added. Similar to the
Sample Definition form, the purpose of the Measurement Definition form is to reduce
typing errors.
Step 4: Entering Measurement Data
The Excel LabData Worksheet should be used to enter the measurement data. You can use
the Show/Hide menu (under Samples and Measurements menu) to reduce the row and
column headings while entering data. Make sure to save your work often to reduce issues
of missing or erroneous data.
5-40
NRM Air v2.2 User Guide
Managing Monitoring Data
You can now sort your data by different columns. To sort, use the Samples -> Select All
menu selection. Then use the normal Sort dialog on the Excel Data tab (Sort & Filter menu
group). Be sure that the “My Data Has Headers” box UNCHECKED. If you cut and paste to
enter data, try using the Paste special variation to paste values only. The Format -> Reapply Formatting menu selection in the Lab Data menu group can be used to re-apply
column data formats.
Step 5: Sending Lab Data to the Forest Service
When sending files to the lab for data input, be sure to only send the Excel file. The helper
database does not store any data. The helper database provides queries and linked tables
to the AirWaterChemistry.mdb file while using the Add-In. The helper database can be
deleted and recreated without impacting the Excel workbook.
NRM Air Add-In Maintenance
Periodically, the Forest Service will provide an update to the Add-In. It is likely that there
will be more frequent updates to the AirWaterChemistry.mdb file that you will need to save
in the Add-In’s installation folder (default location: C:\Program Files\USDA Forest
Service\NRM Air Excel AddIn\Workbook).
You can change the default geographic Extent when new workbooks are enabled with the
Add-In. To do this, run the SetDefaults.exe program located in the Add In’s installed
directory, then select the Extent you desire.
Sending an Up-to-Date Air Water Chemistry Database to the Lab(s)
Use the NRM Workbench to export a new Air Water Chemistry database from the Air
database. After opening the application, select the EXPORT TEMPLATE service. Then
select or type in a geographic to create the correct database workbook. Leave the
Monitoring Project field blank and set the delivery method to “async.” Now click Invoke the
Service to export the updated database.
5-41
NRM Air v2.2 User Guide
Managing Monitoring Data
An email will be sent to you when the new Air Water Chemistry database is ready for
download. Save the .zip file using a name that indicates that it is a BLANK Air Water
Chemistry Template and gives the current date (i.e.,
“BlankAirWaterChemistrydatabase_03-08-2012.zip”). Send this file to the lab(s) with
instructions to extract the content into the installed folder (default location: C:\Program
Files\USDA Forest Service\NRM Air Excel AddIn\Workbook).
Scenario 2: Loading From Excel Lab Data
This scenario presents the steps you will go through when you get data back from a lab.
Step 1: Receiving Lab Data
The first thing to do when you receive a file from a lab is to save it to your local computer.
File naming conventions are “yyymmddProjectNameR#.xlsx”. Be sure to archive the
original files for backup.
Open the Excel workbook from the lab and enable the NRM Air Add-In.
You will see that the workbook already has the LabData Worksheet and custom XML part in
the Excel file. However, you will need to create a new helper database. Do this by using the
Helper Database -> Generate menu selection found in the Control menu group.
5-42
NRM Air v2.2 User Guide
Managing Monitoring Data
Step 2: Using Menu Groups
The NRM Air Excel Add-In menu tab has four primary groupings of functionality: Lab Data,
Control, Analysis, and Server.
•
•
•
•
Lab Data Menu Group
o The Extent is set using the drop-down list in the NRM Air tab
o The Extent influences the choices presented in Lists of Values, Projects, and Sites
o The Extent is saved in the custom XML as part of the Excel Workbook file
Control Menu Group
o If you move the Excel file (and helper database) or lose the helper database, use
either of the following Helper Database menu choices:
Locate to select the new location where the database resides
Generate to create a new helper database
Analysis Menu Group
o The Tabular and Charts menus are available for you to perform QA/QC of the data,
as well as some simple analysis
Server Menu Group
o The Link Tables and Load -> From Excel menu choices will be needed to populate
an empty AirWaterChemistry.mdb database file for loading into the Air Server’s
database
Step 3: Link Tables
Before performing any Add-In functions, make sure the helper database is linked to an upto-date Air Water Chemistry Access database. To do this:
•
•
•
•
Use the Link Tables menu selection from the Server menu group
The lab will have had this set to the file in the installation folder (c:\Program Files\USDA
Forest Service\NRM Air Excel AddIn\Workbook\AirWaterChemistry.mdb)
If that file is up-to-date on your system, it can be used; however, you must still perform the
Link Tables, so click OK
If you exported an Air Water Chemistry database for population after the QA/QC process,
click on “select” to navigate to where that database is located and click on OK to use this
database
5-43
NRM Air v2.2 User Guide
Managing Monitoring Data
•
Make sure you are working with an up-to-date Air Water Chemistry database and that it
was exported from the Production database. A confirmation dialog will be presented after
the tables have been linked, displaying the source and time where and when the export was
performed.
•
Do not continue if the reported database is Test or Development. The timestamp is
provided to help you determine if a new export of the Air Water Chemistry database should
be retrieved.
If the database is reported as Production and is relatively new you can skip the next step.
•
Step 4: Export an Air Water Chemistry Database
You should export a new Air Water Chemistry database after new Monitoring Projects
and/or Monitoring Sites have been created, and after modifications have been made to
Lists of Values (LOVs). The NRM Workbench is used to export a new copy of the Air Water
Chemistry database from the Air database.
1. Select the EXPORT TEMPLATE service from the drop-down list
5-44
NRM Air v2.2 User Guide
Managing Monitoring Data
2. Select or type in the Geographic Extent you are working with
3. Leave the Monitoring Projects field blank and leave the delivery method set to “async”
4. Click “Invoke the Service”
5-45
NRM Air v2.2 User Guide
Managing Monitoring Data
An email will be sent to you when the new Air Water Chemistry database is ready for
download.
1. Save the .zip file, using a name that indicates that it is a BLANK Air Water Chemistry
Template
a. Adding the current date will help you identify how recent your template is
b. for example: “BlankAirWaterChemistryDatabase03082012.zip”
2. Export an Air Water Chemistry database from the zip file for each new Excel
Workbook you are working with
a. Name each export closely to the workbook name or store in the same folder
as the appropriate workbook
b. Name the folder appropriately
3.
Perform Step 3 to link the tables, using the new exported database.
Step 5: QA/QC & Simple Analysis
Data that is entered by the Lab should be delivered in the standard set of measurements,
with any additional measurements added at the end. Before using any of the Analysis
capabilities, please make sure that there are no hidden columns in your worksheet.
Step 6: Use Tabular Menu Group
Via the Tabular menu group, you can add cell formulas, compare replicates, and group
data into standard set format.
The Add Cell Formulas functionality populates cells in the micro-equivalent and
calculation of only columns in the standard set of measurements. Should any cell contain
values provided by the lab, these cells will not be overwritten; instead, this process will
replace cells that contain formulas to allow for repeating the process in cases where “whatif” scenarios would be looked at. Please remember that these formulas do not take into
account any measurements that are not part of the standard set.
5-46
NRM Air v2.2 User Guide
Managing Monitoring Data
The Compare Replicates feature copies the LabData worksheet into a new worksheet and
then sorts the rows by Sample, and removing rows that don’t have replicate samples.
There will now be two rows for each set of replicates and then calculates (based on cell
formula) for average and percent relative standard deviation (%RSD).
The Group Into Standard Set option should not be used with lab data that is already in the
standard set format.
Step 7: Using Charts Menu Group
The Charts menu group provides you with the ability to configure and run chart sets, create
time series plots, and delete chart workbooks.
You can either Configure … new chart sets, or Run … existing chart sets. Using the
Configure … options presents you with a dialog box for modifying or adding new sets of
charts. Chart sets can contain one or many charts. There are three different types of charts
that can be created: scatter charts, histograms with bin count, and box whisker charts. Box
whisker charts provide optional grouping by function; this can be any Excel cell formula,
using “{x}” for the X Axis value.
When creating charts, you can identify the X and Y axis by using column labels (i.e., A, B, C,
… ZZ), or by using measurement parameters. By using measurement parameters, you can
optionally identify the UOM, equipment type, and measurement location as the X or Y axis.
The Run … option allows you to quickly and consistently create charts without having to
input the parameters of the chart each time. When you select Run … you are presented
with the list of chart sets available to choose from. Click in the appropriate box, then click
OK. If desired, you can select more than one chart set to run.
After clicking OK, the Excel Add-In runs a number of processes to create the requested
charts, inserting a new worksheet into the current workbook for each chart created. Chart
columns are searched for and, if both are found, they are copied into the new worksheet. If
5-47
NRM Air v2.2 User Guide
Managing Monitoring Data
a chart column is not found, then a worksheet for that chart is not created. Any rows with
blank values in either column will be removed. If all rows are blank and removed, a chart is
not created. The Add-In runs calculations on the values, then builds and displays the
charts.
To create time series plots, the first step is to select the columns you wish to look at. For
each column that contains a selected cell, a worksheet will be created in your workbook.
The collected sample data (located in column W), is used as the X axis in the chart/plot. A
Scatter Chart is then created using the same methods used by the Run … option of a chart
set.
Finally, you have the ability to delete all of the worksheets in your Excel workbook that
include chart objects on them.
Step 8: Load from Excel
Once the QA/QC work is completed, save a copy of the Excel file as a backup. Then remove
all measurement columns you don’t want to have loaded into the Air database. Use the
Link Tables functionality to make sure you are using an up-to-date Air Water Chemistry
database—this database is populated with the lab data from the worksheet.
To load the Air Water Chemistry database, use the Load -> Load from Excel option. This
creates a process that looks at the data in the worksheet and produces a “Pre-Load Report”
that identifies any issues by row. Issues include such things as identification of missing and
required data, and these issues should be resolved before continuing with the load
procedure. After completing the load successfully, the Air Water Chemistry database now
contains the data from the LabData worksheet.
Step 9: NRM Workbench, Validate, & Load
The NRM Workbench is used to validate the Air Water Chemistry database. You will want
to review the validation report from the Workbench to determine what will occur during
the load; if the validation report identifies any problems, you may need to go back to the
workbook to make some adjustments.
5-48
NRM Air v2.2 User Guide
Managing Monitoring Data
After reviewing the validation report, make any necessary adjustments to the workbook,
then use the Excel Add-In Load -> from Excel menu selection again. You can reuse the same
Air Water Chemistry database, as it was previously cleaned before being populated.
After reviewing the new validation report and determining that it is accurate and that no
errors exist, use the NRM Workbench to Load the Air Water Chemistry database into the
Air database. This Load procedure will create a report that will be delivered either
synchronously or asynchronously, depending on your selection.
It is possible that errors will occur during the Load process. If these data issues can be
corrected, you should make these corrections in the workbook and then use the NRM
Workbench to re-validate the data to determine if the corrections will be made. If the
corrections will be made, use the NRM Workbench again to Load and make the corrections.
5-49
NRM Air v2.2 User Guide
Managing Monitoring Data
If you encounter Validation or Load errors that may not be caused by data error, please be
sure to report the problem to the helpdesk, and attach the validation and/or load report(s)
to the helpdesk ticket.
Scenario 3: Load from NPSTORET Back End
This scenario outlines the steps necessary to load NPSTORET data into the Air database.
Remember that you can only load data for one Monitoring Project at a time; if you have
multiple Monitoring Projects, you will need to load each Project separately. Also, you will
save time if you make sure all Monitoring Projects you are working with exist in the Air
database before attempting to load.
Step 1: Receiving a New NPSTORET Back End
When new NPSTORET data is received, create a new Excel file on your local hard drive. File
naming conventions are “yyymmddProjectNameR#.xlsx”. Open the new Excel workbook
and enable the NRM Air Add-In. Remember that while the Add-In is enabled, all workbooks
opened within the same Excel application will be enabled. This means that an Access
database will be copied to the same location as all Excel files you open while the Add-In is
enabled; this file will use the name of the Excel file as a prefix to “Helper.accdb.”
Additionally, a LabData worksheet will be added to each workbook, and a custom XML file
will be added to the same folder as the workbook to manage settings (i.e., Extent). If you
open additional instances of the Excel application, these will not be affected by the Add-In,
unless you enable it there as well.
5-50
NRM Air v2.2 User Guide
Managing Monitoring Data
Step 2: Using Menu Groups
In the Lab Data menu group, you will want to set the Extent to Eastern (the Eastern extent
is used for all NPSTORET data). Setting the correct Extent is important because it
determines what data is included in the Lists of Values (LOVs). The Extent is stored in the
custom XML as part of the Excel workbook file.
If you move the Excel file (and/or helper database), or lose the helper database, you will
need to use the Locate … and Generate functions under the Control menu group. The
Locate … function is used to select the new location where the database resides. The
Generate function creates a new helper database.
The Tabular and Charts options in the Analysis menu group help you perform QA/QC of
the data, as well as some providing some simple analysis tools.
From the Server menu group you will use the Link Tables and Load -> From Excel …
menu choices. These functions are needed to populate an empty AirWaterChemistry.mdb
database for loading into the Air server’s database.
Step 3: Link Tables
Before performing any Add-In capabilities, make sure that the helper database is linked to
an up-to-date Air Water Chemistry Access database.
To do this:
•
•
•
•
Use the Link Tables menu selection from the Server menu group
The lab will have had this set to the file in the installation folder (c:\Program Files\USDA
Forest Service\NRM Air Excel AddIn\Workbook\AirWaterChemistry.mdb)
If that file is up-to-date on your system, it can be used; however, you must still perform the
Link Tables, so click OK
If you exported an Air Water Chemistry database for population after the QA/QC process,
click on “select” to navigate to where that database is located and click on OK to use this
database
5-51
NRM Air v2.2 User Guide
Managing Monitoring Data
•
Make sure you are working with an up-to-date Air Water Chemistry database and that it
was exported from the Production database. A confirmation dialog will be presented after
the tables have been linked, displaying the source and time where and when the export was
performed.
•
Do not continue if the reported database is Test or Development. The timestamp is
provided to help you determine if a new export of the Air Water Chemistry database should
be retrieved.
If the database is reported as Production and is relatively new you can skip the next step.
•
Step 4: Export an Air Water Chemistry Database
You will want to make sure you export an updated Air Water Chemistry database after new
Monitoring Projects and/or Monitoring Sites are created, and after modifications have been
made to Lists of Values (LOVs). Use the NRM Workbench to export a new Air Water
Chemistry database from the Air database:
1. Select the EXPORT TEMPLATE service
2. Select or type in an Geographic Extent (“Eastern” is always used for NPSTORET
data)
3. Leave the Monitoring Projects field blank, and set delivery method to “async”
4. Click on the Invoke the Service button to export the template; because you selected
“async” delivery, you will receive an email with a link to the exported file
When you receive the email, download the zip file and save it to your hard drive using a
filename that indicates that it is a BLANK Air Water Chemistry template and indicates the
date it was created (i.e., “BlankAirWaterChemistrydatabase_03-08-2012.zip”). You will
want to export an Air Water Chemistry database from this zip file for each new Excel
workbook you will be working with, and use a name that closely matches the workbook
name or store the file (together with the workbook) in an appropriately named folder.
Perform Step 3 to link the tables, using the new exported database.
Step 5: Load from NPSTORET
Before data from the NPSTORET Back End can be brought into the Excel workbook for
analysis, it must first be loaded into the Air Water Chemistry database. Use the Link Tables
functionality to make sure you are using an up-to-date Air Water Chemistry database—this
database will be populated with the lab data from the NPSTORET Back End database.
5-52
NRM Air v2.2 User Guide
Managing Monitoring Data
Use the Load -> Load from NPSTORET menu selection to load the Air Water Chemistry
database. The Add-In will present four different dialog boxes where you are asked to link
to files and other information necessary for the Load you are currently performing.
The first dialog box is the NPSTORET Link Database Tables window, which identifies which
files should be used for the Back End Database and the Definition Table Database. Clicking
on the select … button opens a Windows Select File dialog box.
When you click OK in the NPSTORET Link Database Tables window, the Record Selection
Dialog box automatically opens. In the upper left section of the window is information
letting you know that the tables you selected in the previous window have been linked.
This area also gives you some instructions about what to do in this form. At this point, you
will want to select all of the Site Visits you want included in this Load. This part of the
window also lets you know that the following process could take some time to complete.
After you have selected the Monitoring Visit Dates of interest, click on OK. The Record
Selection Dialog window now presents a list of Monitoring Project names for you to select
from—this identifies what Protocol was used in collecting the sample. After selecting the
correct Monitoring Project, click OK.
5-53
NRM Air v2.2 User Guide
Managing Monitoring Data
You will receive a confirmation message, letting you know that all of the measurement
parameters have been defined. Click on OK to go to the next step.
The Add-In continues to process your Load setup. You will now receive confirmation that
all of the analysis procedures and equipment types have been defined. Again, click on OK
to proceed.
The Add-In then interacts with the Access database to populate it with data from the
NPSTORET back end. Please note that once you start this process you are unable to cancel
out of it. This window will remain on your screen until the process has finished.
5-54
NRM Air v2.2 User Guide
Managing Monitoring Data
Upon successful completion of the previous process, you will receive a notice letting you
know this has finished and where the resulting file is located. Click on OK to complete this
process.
Step 6: QA/QC & Simple Analysis
The Tabular and Charts menus provide you with the tools you need to perform QA/QC and
simple analysis functions on your data. Using the Tabular menu option you can Compare
Replicates. This functionality copies the LabData worksheet to a new worksheet, then
sorts the rows by Sample and removes any rows that don’t have replicate samples. The
Add-In then adds two rows for each set or replicates and, by cell formula, calculates
Average and Percent Relative Standard Deviation (%RSD).
The Charts menu gives you the ability to Configure … and Run … chart sets, create Time
Series Plots, and delete chart worksheets. The Configure … option presents a dialog for
modifying or adding new sets of charts. Each chart set contains one or more pre-defined
charts. Chart types include: Scatter, Histogram (with bin count), and Box Whisker (with
optional grouping by function). Box Whisker charts provide optional grouping by function;
this can be any Excel cell formula, using “{x}” for the X Axis value.
When creating charts, you can identify the X and Y axis by using column labels (i.e., A, B, C,
… ZZ), or by using measurement parameters. By using measurement parameters, you can
optionally identify the UOM, equipment type, and measurement location as the X or Y axis.
Choosing the Run … option prompts you to select which chart set(s) to run and then the
Add-In produces a new worksheet for each chart selected. Chart columns are searched for,
and if found they are copied into the new worksheet; if a chart column is not found, the
worksheet for that chart is not created. If blank values are found in either column, that row
is removed from the chart; and if all rows are removed (i.e., they are all blank), the chart is
not created. The Add-In then performs calculations on the values available and then the
corresponding chart is created on that worksheet.
Time Series Plots allow you to select specific columns to run against. For each column that
has a cell selected, a worksheet is produced. The collected sample data (in column W) is
used for the X axis of the chart. Finally, a Scatter Chart is created, using the same method
used by the Run … process of a chart set.
You can use the Delete All Chart Worksheets option to delete all of the worksheets in
your current workbook that contain charts on them.
5-55
NRM Air v2.2 User Guide
Managing Monitoring Data
Step 7: NRM Workbench – Validate & Load
The NRM Workbench is used to validate the Air Water Chemistry database. You will want
to review the validation report from the Workbench to determine what will occur during
the load; if the validation report identifies any problems, you may need to go back to the
workbook to make some adjustments.
After reviewing the validation report, make any necessary adjustments to the workbook,
then use the Excel Add-In Load -> from Excel menu selection again. You can reuse the
same Air Water Chemistry database, as it was previously cleaned before being populated.
After reviewing the new validation report and determining that it is accurate and that no
errors exist, use the NRM Workbench to Load the Air Water Chemistry database into the
Air database. This Load procedure will create a report that will be delivered either
synchronously or asynchronously, depending on your selection.
5-56
NRM Air v2.2 User Guide
Managing Monitoring Data
It is possible that errors will occur during the Load process. If these data issues can be
corrected, you should make these corrections in the workbook and then use the NRM
Workbench to re-validate the data to determine if the corrections will be made. If the
corrections will be made, use the NRM Workbench again to Load and make the corrections.
If you encounter Validation or Load errors that may not be caused by data error, please be
sure to report the problem to the helpdesk, and attach the validation and/or load report(s)
to the helpdesk ticket.
Scenario 4: Analyze Historical Lab Data
The Air database contains many years of collected data that you can use in analysis. This
Scenario takes you through pulling the data out of the database and using functionality
from the Excel Add-In to analyze.
5-57
NRM Air v2.2 User Guide
Managing Monitoring Data
Step 1: Export an Air Water Chemistry Database
You should export a new Air Water Chemistry database after new Monitoring Projects
and/or Monitoring Sites have been created, and after modifications have been made to
Lists of Values (LOVs). The NRM Workbench is used to export a new copy of the Air Water
Chemistry database from the Air database.
1. Select the EXPORT TEMPLATE service from the drop-down list
2. Select or type in the Geographic Extent you are working with
5-58
NRM Air v2.2 User Guide
Managing Monitoring Data
3. Select the Monitoring Project(s) you want to export the data from
4. Select the Site Project CNs you want to export the data from
5.
Leave the delivery method set to “async”
5-59
NRM Air v2.2 User Guide
Managing Monitoring Data
6. Click “Invoke the Service”
An email will be sent to you when the new Air Water Chemistry database is ready
for download.
7. Export the AirWaterChemistry.mdb Access database, appending something that
indicates the selection of monitoring project(s) made, for example:
a. “AirWaterChemistryR5LakesLT_20100409.mdb”
5-60
NRM Air v2.2 User Guide
Managing Monitoring Data
Step 2: Using Excel NRM Air Add-In with Historical Data
When working with historical data in the Excel Add-In, you will start by creating a new
Excel file on your local PC; (naming convention), then enable the NRM Air Excel Add-In.
Please remember that while the Add-In is enabled, *all* of the workbooks opened within
the same Excel application will be enabled. You can start other instances of the Excel
application and these will not be affected by the Add-In, unless you enable it in those
instances as well.
Having the Excel Add-In enabled means that an Access database will be copied into the
same folder location as the Excel file and will be named with the Excel file name as a prefix
to “Helper.accdb.” Additionally, a LabData worksheet will be added to your workbook.
Finally, a custom XML file is also added to the workbook to manage settings such as Extent.
Step 3: Menu Groups
The NRM Air Excel Add-In menu tab has four primary groupings of functionality: Lab Data,
Control, Analysis, and Server.
•
•
•
Lab Data Menu Group
o The Extent is set using the drop-down list in the NRM Air tab
o The Extent influences the choices presented in Lists of Values, Projects, and Sites
o The Extent is saved in the custom XML as part of the Excel Workbook file
Control Menu Group
o If you move the Excel file (and/or helper database) or lose the helper database, use
either of the following Helper Database menu choices:
Locate to select the new location where the database resides
Generate to create a new helper database
Analysis Menu Group
o The Tabular and Charts menus are available for you to perform QA/QC of the data,
as well as some simple analysis
5-61
NRM Air v2.2 User Guide
•
Managing Monitoring Data
Server Menu Group
o The Link Tables and Load -> From Excel menu choices will be needed to populate
an empty AirWaterChemistry.mdb database file for loading into the Air Server’s
database
Step 4: Link Tables
Before performing any Add-In functions, make sure the helper database is linked to an upto-date Air Water Chemistry Access database. To do this:
•
•
Use the Link Tables menu selection from the Server menu group
Make sure you select the Air Water Chemistry database you exported with the Project(s)
selection
•
Make sure you are working with an up-to-date Air Water Chemistry database and that it
was exported from the Production database. A confirmation dialog will be presented after
the tables have been linked, displaying the source and time where and when the export was
performed.
•
Do not continue if the reported database is Test or Development. The timestamp is
provided to help you determine if a new export of the Air Water Chemistry database should
be retrieved.
Step 5: Retrieve Data from the Air Water Chemistry Database
Use the Retrieve -> All … menu selection. This loads data from the Air Water Chemistry
database into the Lab Data worksheet. Sample rows are loaded first, followed by the Field
5-62
NRM Air v2.2 User Guide
Managing Monitoring Data
Measurement columns, and finally Lab Measurement columns are loaded. At this point, the
data can be checked and analyzed.
Step 6: QA/QC & Simple Analysis
The Tabular and Charts menus provide you with the tools you need to perform QA/QC and
simple analysis functions on your data. Using the Tabular menu option you can Compare
Replicates. This functionality copies the LabData worksheet to a new worksheet, then
sorts the rows by Sample and removes any rows that don’t have replicate samples. The
Add-In then adds two rows for each set or replicates and, by cell formula, calculates
Average and Percent Relative Standard Deviation (%RSD).
The Charts menu gives you the ability to Configure … and Run … chart sets, create Time
Series Plots, and delete chart worksheets. The Configure … option presents a dialog for
modifying or adding new sets of charts. Each chart set contains one or more pre-defined
charts. Chart types include: Scatter, Histogram (with bin count), and Box Whisker (with
optional grouping by function). Box Whisker charts provide optional grouping by function;
this can be any Excel cell formula, using “{x}” for the X Axis value.
When creating charts, you can identify the X and Y axis by using column labels (i.e., A, B, C,
… ZZ), or by using measurement parameters. By using measurement parameters, you can
optionally identify the UOM, equipment type, and measurement location as the X or Y axis.
Choosing the Run … option prompts you to select which chart set(s) to run and then the
Add-In produces a new worksheet for each chart selected. Chart columns are searched for,
and if found they are copied into the new worksheet; if a chart column is not found, the
worksheet for that chart is not created. If blank values are found in either column, that row
is removed from the chart; and if all rows are removed (i.e., they are all blank), the chart is
not created. The Add-In then performs calculations on the values available and then the
corresponding chart is created on that worksheet.
Time Series Plots allow you to select specific columns to run against. For each column that
has a cell selected, a worksheet is produced. The collected sample data (in column W) is
used for the X axis of the chart. Finally, a Scatter Chart is created, using the same method
used by the Run … process of a chart set.
You can use the Delete All Chart Worksheets option to delete all of the worksheets in
your current workbook that contain charts on them.
5-63
NRM Air v2.2 User Guide
Managing Monitoring Data
ADDITIONAL INFORMATION
Post-Installation (Non-Lab)
In order to perform the loading of data into an AirWaterChemistry.mdb database, the
folders where the helper databases will be saved are required to be “trusted.” Use Access
to add the folders to the Trusted Locations list.
Access and User Interface
The Excel NRM Add-In uses Access in the background and tells Access not to display its
user interface. If you have an Access application open before enabling the Excel Add-In,
there is no impact to that instance of the Access application.
If you open a new Access application after the Add-In has been enabled, Access will not
display. In order to see a new Access application after the Excel Add-In has been enabled,
exit the Excel application that has the NRM Add-In enabled.
Office Excel 2010 File Content
The Excel file (.xslx) is now a collection of XML files. The NRM Add-In adds a “custom XML
part” to this collection that contains information specific to each document such as: Extent;
file paths to the helper, AirWaterChemistry, and NPSTORET databases; and chart sets.
Excel & Custom Content
Excel allows you to remove custom content. To remove custom content, click on the File
menu, then select Prepare for Sharing, then Check for Issues -> Inspect Document. By
removing this content, the Extent that was used to populate the LabData worksheet will be
removed, as well as the rest of the information held in the custom XML part.
5-64