SLOCUM AUVG GLIDER CODE
The brain of the SLOCUM AUVG is a Persistor Instruments Inc. CF1 computer
chip, which is based on Motorola’s MC68CK338 design. The disk space is
provided via a SanDisk removable compact flash card (www.sandisk.com,
www.persistor.com/pii/products/cf.htm). The SLOCUM vehicle also contains a
separate Persistor for the collection and logging of scientific data. The following
is a discussion of the software design as it pertains to the operation of the glider.
Topics included in this discussion are:
1.
2.
3.
4.
5.
6.
7.
Glider Operating Systems
Slocum Directory Structure
Mission File Structure and Glider Control
Data Retrieval
Data Processing
Glider Simulations
Real World Glider Deployments
OPERATING SYSTEMS
The glider source code is written and maintained by Dinkum Software and Webb
Research Corporation and stored on a server at Webb Research Corporation in
Falmouth, MA. The code was originally developed by the AUV Laboratory at MIT
SeaGrant (auvlab.mit.edu). The code was originally developed to control the
Odyssey series of MIT auv’s, but has since been adapted for exclusive use on
the Webb AUVGs. The source code is available via an internet accessible
WinCVS database, which allows for extraction of current or older revisions.
Access to this WinCVS database must be pre-arranged.
Three pieces of software, a boot monitor and two operating systems, are used for
the development and control of the Slocum vehicle:
1) The PicoDOS Boot Monitor (PBM) is used to boot PicoDOS. PBM is the first
bit of software to execute from a power-up or reset. Its primary job is to figure out
what program should take over at reset (PicoDos or GliderDos) and start it
running. PBM also initializes the hardware to enable serial communications and
to prevent the watchdog timer from interfering with the monitor.
2) The resident operating system for the Persistor CF1 is PicoDOS (Persistor
Instruments Card Or Disk Operating System). PicoDOS is a smaller version of
DOS developed exclusively for use with the Persistor CF1 & CF2 computer
systems. Uploading of new Glider Controller code and new mission files are
performed while in PicoDOS. Typing help <> will display the available PicoDOS
commands.
3) GliderDOS is an application that is loaded onto the Persistor. This operating
system is a superset of PicoDOS and executes the glider missions once they
have been written and uploaded to the glider vehicle. It is written in C/C++ and
compiled using Metrowerks CodeWarrior, and then postlinked and uploaded to
the SLOCUM vehicle via MotoCross. Typing help <> wil display the available
GliderDOS commands.
NAVIGATING BETWEEN PICODOS & GLIDERDOS
PicoDOS is the operating system which has been developed for and is shipped
with the Persistor CF1. There are a few reasons to work in PicoDos. They
include loading new source code for the glider (GliderDOS), working in the file
structure without the device drivers being called, and loading new files to the
glider. When configured to boot into GliderDOS and once communication has
been established, the glider will run autoexec.mi, a file containing settings
specific to that glider. Enter the following to exit GliderDOS and get to PicoDOS:
Press CTRL C to stop the mission and return to GliderDOS
exit pbm <>
pico <>
To return to GliderDOS, enter the following:
app <>
This will cause GliderDOS to boot and call a file named autoexec.mi, which has
all of the glider’s calibration coefficients and other various settings. The
autoexec.mi file is unique to each glider and should not be interchanged between
gliders.
In certain situations, which will be discussed later, it will be necessary to instruct
the glider to boot up into either PicoDOS or GliderDOS. For example, after you
have downloaded, compiled, and linked new glider code, the next step is to
upload that new code to the glider. Should an error occur during the code upload
resulting in an endless loop or the code hanging, if the glider is configured to boot
into PicoDOS, it may be turned off and then on to reset. If the glider is configured
to boot into GliderDOS, however, you will have to manually reset the glider as it
will continually try to boot the faulty code. To configure the glider to boot into
PicoDOS upon power up, enter the following before powering attempting to load
new code:
boot pico <>
To boot into GliderDOS upon power up, enter the following before powering
down or resetting:
boot app <>
After a mission has been completed and/or a CTRL+C has been executed to
return to GliderDOS, you must exit GliderDOS before powering down. This is
accomplished by entering the following:
exit <>
It is now safe to power down the glider.
GLIDER CODE DESIGN
The glider receives misison instructions through user created text based mission
files (.mi) loaded onto the Persistor. These missions are then executed from
within GliderDOS. Mission files are based on a layered single thread approach
where tasks are coded into behaviors (behavior:_) which themselves, are
composed of behavior arguments (b_arg:_) and sensor values (sensor:_). A
sensor is any variable whose value is changing during the course of a mission.
All of the sensors have prefixes attached to them that describe how they are
used or who uses them in a mission. The prefix definitions are as follows:
m_sensor
c_sensor
u_sensor
f_sensor
x_sensor
s_sensor
measured by the glider
commanded by the user
user defined before runtime
factory setting
computed at runtime by the glider software
simulated variable used only in glider simulation mode
Some examples of sensor values are gps fixes, piston displacement pump
position and CTD values. Every value (either scientific or diagnostic) that is
recorded is referred to as a sensor. All sensor values, behavior arguments and
behaviors are defined in a file called MASTERDATA, located on the computer in:
c:\mycode\glider\code\
The MASTERDATA file becomes part of the compiled glider code which runs the
glider misisons. When a mission is run, the glider checks the sensor values
against those in MASTERDATA. If a sensor value is set in the mission file, it
overrides the default value in the MASTERDATA; otherwise, the default value is
used to determine the behavior or movement of the glider.
SLOCUM DIRECTORY STRUCTURE
Structured in PicoDOS, but accessible in PicoDOS and GliderDOS, are the
following directories:
BIN
CONFIG
LOG
MAFILES
MISSION
SENTLOGS
STATE
AUTOEXEC.BAT
<DIR>
<DIR>
<DIR>
<DIR>
<DIR>
<DIR>
<DIR>
Each folder in this directory is allocated a finite amount of memory in which it can
store files. This point is particularly important in the case of the SENTLOG folder
and the consequences will be discussed during the discussion of this folder. The
contents and functions of each folder are as follows:
BIN folder: contains picoDOS executable files (.run), all of which are run while in
PicoDOS. The contents include:
ADTEST.RUN: displays and updates raw voltages for Analog to Digital
(A>D) devices. Used during calibration procedures.
ALLOFF.RUN: is placed in autoexec.bat to zero all registers and turn on
the RF modem to establish communications. It is EXTREMELY
important that a copy of this file is in this directory at all times. If the
glider reboots/resets into PicoDOS and alloff.run is not in the directory,
you will not be able to establish communications in order to enter
GliderDOS and run a mission. In other words, get the boat ready, cause
you’ve got to go out to find the glider and manually reset it.
CONSCI.RUN: swithces the RF modem over to the science computer to
allow direct access to download data files as well as perform code loads.
SRTEST.RUN: allows for switching on/off of selected bits. Used for
hardware board interfacing.
TALK.RUN: ports out to specific components such as CTD, ARGOS,
attitude sensor, and GPS for direct access and setup.
UARTTEST.RUN: testing of UART drivers for programming.
ZR.RUN: allows the transferring of files to the glider via zmodem, which is
a more robust communications protocol than x or y modem.
ZS.RUN: allows the transferring of files from the glider to a shoreside
computer.
CONFIG folder: contains files which set the glider configuration specifically for
that glider. The contents include:
CONFIG.SCI: specify which sensor values are sent to the glider and
when. While multiple instruments can be loaded onto the glider,
modification of this file instructs the glider to put various instruments in or
out of service.
SBDLIST.DAT: specify which sensors are recorded in the short binary
data (.sbd) files. This file type, as well as others, will be introduced in
the discussion of the LOG folder. SBDLIST.DAT can be modified to
include logging of critical data only, in order to decrease the data
transmission time and, thus, the time the glider is on the surface.
AUTOEXEC.MI: glider specific file which contains the calibration
constants, factory settings and serial and part numbers for glider
hardware. Also includes iridium phone numbers. This file is read by the
glider application (GliderDOS) upon startup of the system.
SIMUL.SIM: converts the glider to a benchtop simulator.
LOG folder: contains all mission derived data files. Missions are uniquely
numbered and are divided into segments. Data files are named according to the
8.3 DOS naming convention:
mmmmssss.XXX
Where mmmm is the unique mission number and ssss is the particular segment
for that mission. XXX refers to one of the following three file types:
.DBD: dinkum binary data files. These binary files store the values of all
active sensors for a particular mission. The size of these files is
dependent on a number of factors including number of sensors being
recorded as set in CONFIG.SCI and the length of time between
surfacings as set in that particular mission file. Each DBD file for a
particular mission represents a segment of that mission. A new mission
segment is created each time the glider surfaces or dives.
Once the data files have been transmitted back to shore, a number of
DOS executables have been written which rename the files to the
following format:
GGGG_YYYY_DDD_MM_SS
Where GGGG is the name of the glider, YYYY is the year of the mission,
DDD is the julian day of the mission, MM is the mission number, and SS
is the segment number of that mission. From here, another executable
converts the binary files to ASCII files, which can be processed any
number of ways.
With the large number of sensors that the glider uses to control it’s
movement and scientific data acquisition, it becomes apparent that these
DBD files can become extremely large, even in binary form. This means
that more time is needed to transmit them back to shore, which means
the glider is on the surface longer, subjecting it to possible damage.
.SBD: short binary data files. These binary files contain sensor values
recorded as specified in the SBDLIST.DAT file. These files can then be
transmitted back to shore, significantly reducing data transmission time
and, ie: surface interval time.
.MLG: mission log files. These text files keep track of all calls for
behaviors and device drivers. In other words, the file is a record of
everything the glider does while under water and on the surface. These
files are very useful for diagnosing problems with glider operation and/or
reasons for abort sequences.
After a .dbd, .sbd or .mlg file is sent to shore, it is removed from the LOG folder
and placed to the SENTLOGS folder.
MAFILES folder: contains ma files (.ma). These files contain waypoint
coordinates or specific conditions for the activation of behaviors. The glider will
open and read these files only if instructed in the mission file to do so. The
specific b_arg used is:
b_arg: args_from_file(enum) N
A practical use of this feature is to write a semi-generic mission and to insert the
the behavior argument b_arg: args_from_file in the behavior: goto_list. When
activated, this behavior instructs the glider to read the specific .ma file in the
MAFILES folder containing the waypoints. The use and versatility of .ma files will
be discussed much more extensively.
MISSION folder: contains the glider mission files (.mi). These files are written
onshore and then uploaded to the glider. Mission files are constructed by
stringing together sensor values (sensor:_) and behaviors (behavior:_) to
instruct the glider in how to perform various tasks including, but not limited to,
vertical and horizontal movement, logging of scientific data, and surfacing and
data transmission intervals. The theory and process of writing mission files will
be discussed in further detail in a subsequent section of this manual.
SENTLOGS folder: contains the .dbd, .sbd, and .mlg files once they have been
transmitted back to shore. Each time the glider surfaces and transmits files to a
host computer, it removes the file from the MISSION folder and places the file in
this folder, where it resides until the mission has been completed. After the
mission has been completed, all of the files in this folder may be deleted. For
this reason, it is fairly important that the contents of this folder are always erased
between missions. If they are not and another mission is run, you run the chance
of filling up this folder with new and old data files. If this in the middle of a
mission, the glider will abort and go to the surface, at which time the folder will
need to be emptied and the mission started over.
AUTOEXEC.BAT: typically has only one line which calls ALLOFF.RUN. If the
glider boots up into PicoDOS, ALLOFF.RUN zeroes all registers and turns on the
RF communication so that contact can be established with the glider.
UNDERSTANDING & WRITING SLOCUM MISSIONS
During a typical glider deployment, a text-based mission file is created by the
user and uploaded to the glider. When commanded to do so, the glider reads the
mission file which sets parameters to obtain the desired glider path, vertical
movement and data acquisition products. The following is a brief discussion of
the structure of the mission files as well as any ancillary files used to determine
the glider’s mission. There are two file types that are used for the execution of a
mission.
.MI files: also referred to as mission files or missions (.mi). These files
consist of sensor values (sensor: ), behavior arguments (b_arg: ) and
behaviors (behavior: ) listed such that they determine the glider’s
behavior while underwater and on the surface.
.MA files: these files are called by the mission files during mission
execution and can contain behaviors or waypoint lists that are specific
to that mission.
In addition to these two filetypes, a file called MASTERDATA (a copy of whichi is
located in Appendix A of this manual) contains all of the sensor definitions and
behavior arguments that are called in each behavior. This file is located in the
WinCVS directory:
glider/code/
Once the mission file has been created, it must be uploaded to the glider before it
can be executed. This is accomplished using zmodem, a file transfer protocol
which features error checking and crash recovery protection. In addition to these
improvements, zmodem transmits data in 512 byte blocks resulting in faster data
transfer rates. Once the glider has been powered up, launch ProComm and start
up PicoDOS. At the C:\ prompt, enter the following:
zr <>
This sets the glider to zmodem receive mode. Next, go to the DATA menu bar
and choose send file. A file selection window will appear. Select the mission file
and hit OK. A progress bar will appear and the file will be uploaded to the glider
MISSION folder. It’s important to remember that mission filenames must be 8
characters or less, not including the .mi extension, otherwise you won’t be able to
upload them.
It is also possible to transfer any file from the glider to a shoreside computer
using zmodem. To do this, enter the following at the C:\ prompt:
zs filename.ext <>
where filename.ext is the file (with appropriate extension) you want to transfer.
Once the mission has been uploaded to the glider, you must instruct the glider to
execute it. This is done from within GliderDOS. Switch over to GliderDOS using
the app <> command. The glider will execute autoexec.mi which zeroes all
registers and turns on the RF modem. Then, it will execute status.mi, which
samples all sensors, makes sure the glider is at the surface and then exits to the
GliderDOS prompt.
To run a mission, at the GliderDOS prompt, enter
run mission.mi <>
where mission.mi is the name of the mission to be executed. The glider will
begin to execute the mission.
Once the glider has been instructed to execute a mission, GliderDOS reads the
mission and assigns a priority to each behavior. The priority is denoted by a
numerical assignment and is determined by the physical location of the behavior
in the mission text file. The assigned priority list is called a log_c_stack or stack
and resembles the following:
log_c_stack():
1 – abend
2 – surface
3 – set_heading
4 – yo 5 – prepare_to_dive
6 – sensors_in
where the words following each number are specific behaviors as defined in that
specific mission. When one behavior is assigned priority over another and a
behavior argument is satisfied in both of the behaviors (ie: if two or more surface
behaviors have been written into the mission), only the behavior with the higher
priority is executed.
After the log_c_stack() has been created, GliderDOS begins to scroll through the
mission, activating various sensors and/or initiating behaviors according to the
behavior arguments. Whether a particular behavior changes from one state to
another depends upon numerous sensor values and behavior arguments. While
a mission is being executed, all assigned behaviors are in one of the following
states:
(1) Uninitiated
(2) Waiting for Activation
(3) Active
(4) Complete
Towards the end of MASTERDATA is a list of numbers and their ‘assigned’
actions. This list is named ‘beh_args.h’ and can be found in the C code. This list
primarily deals with the two b_arg statements: b_arg: start_when and
b_arg:stop_when, and determines when a particular behavior changes from
one state to another. Only one behavior can be active at a time.
GLIDER CONTROL
Glider control is organized into 5 layers. These layers are visualized in the
following hierarchal structure:
Layered Control (behaviors)
Dynamic Control (movement of motors)
Device Drivers/Device Scheduler
Sensor Processing Algorithms
Data Logging (.dbd, .sbd, .mlg, .log files)
Layered Control determines which behavior is controlling glider movement.
Once layered control determines this, Dynamic Control moves the motors,
inflates/deflates the bladders, etc., to initiate the movement or behavior
commanded under the layered control. In order to do this, Dynamic Control uses
a specific set of Device Drivers to perform the movements. Sensor Processing
involves the algorithm calculations to assign values (1 or 0) to the sensors. Log
Data is self explanatory, except that values of all sensors (ie: variables),
instruments, gps fixes, etc are actually logged.
Missions are executed through layered control, which is defined by the behaviors
and their organization in the stack for a specific mission. In other words, layered
control utilizes motor movements (dynamic control), device drivers and
schedulers, sensor processing algorithms and data logging to achieve the
desired mission.
In order to understand the lower levels of control, it is useful to look at how
various types of abort sequences are initiated and which control layers are used
to achieve the aborts. There are three kinds of aborts which the glider can
trigger to get to the surface: synchronous abort, out of band abort and a
hardware generated abort. Each type of abort utilizes different control layers
during the abort. A much more thorough and in-depth discussion of abort causes
and sequences is located at the back of this manual (Appendix B) as well as in
the WinCVS directory:
/glider/doco/abort-sequences.txt
A synchronous abort is an abort in which the behaviors selected in the mission
files are no longer called. Synchronous aborts are the most common type of
abort encountered during mission execution. The most common cause is an
incorrectly configured mission file. During a synchronous abort, the behaviors
are no longer driving the glider. Instead, Dynamic Control is used to bring the
glider to the surface. The layers used
Dynamic Control (movement of motors)
Device Drivers
Sensor Processing Algorithms
Data Logging (.dbd, .sbd, .mlg, .log files)
Under dynamic control, the glider software uses the device drivers to move
motors so that positive buoyancy is achieved. All communication (RF modem,
iridium, ARGOS) and location (GPS) devices are also turned on. Also, all system
log files are available to trace the cause of the abort. The current depth of the
glider is reported and the raw gps information is reported to aid in location of the
wayward glider.
The abort terminates when the glider detects that it is on the surface or if the user
interrupts with a keystroke (CTRL + C). Once the abort is terminated, control is
returned to GliderDOS. After a synchronous abort, a prompt similar to:
GliderDos A # > will appear. The A stands for abort (N stands for mission ended
normally and I stands for intial, ie: no mission has been run. ) and the # refers to
a specific abort code. The following is the complete list of abort codes:
-3:
-2:
-1:
MS_NONE
MS_COMPLETED_ABNORMALLY
MS_COMPLETED_NORMALLY
0:
1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
MS_IN_PROGRESS
MS_ABORT_STACK_IS_IDLE
MS_ABORT_HEADING_IS_IDLE
MS_ABORT_PITCH_IS_IDLE
MS_ABORT_BPUMP_IS_IDLE
MS_ABORT_THRENG_IS_IDLE
MS_ABORT_BEH_ERROR
MS_ABORT_OVERDEPTH
MS_ABORT_OVERTIME
MS_ABORT_UNDERVOLTS
MS_ABORT_SAMEDEPTH_FO
MS_ABORT_USER_INTERRUPT
MS_ABORT_NOINPUT
MS_ABORT_INFLECTION
MS_ABORT_NO_TICKLE
MS_ABORT_ENG_PRESSURE
MS_ABORT_DEVICE_ERROR
MS_ABORT_DEV_NOT_INSTALLED
MS_ABORT_WPT_TOOFAR
The specific reason for the abort is also recorded in the mission log file for postmission assessment. Synchronous aborts are often triggered by the behavior
abend. Behavior abend is one of, if not the most, high priority behavior and is
usually placed at the top of the mission stack. Included within abend are a
number of processes which are monitored for irregularities. Below is a list of
abort codes relating to abend. For an explanation, see Appendix A:
MS_ABORT_USER_INTERRUPT: user keystrok interrupt.
MS_ABORT_OVERDEPTH: glider has gone below the maximum working
depth as set by b_arg: overdepth(m).
MS_ABORT_OVERTIME: the mission has lasted longer than b_arg:
overtime(sec).
MS_ABORT_UNDERVOLTS: the battery voltage is less than b_arg:
undervolts(volts).
MS_ABORT_NOINPUT: a required input wasn’t sampling as it should be.
MS_ABORT_NO_TICKLE: the hardware watchdog was not tickled.
MS_ABORT_WPT_TOOFAR: the maximum allowable distance as set by
b_arg: max_wpt_distance(m) has been exceeded.
Another cause of a synchronous abort is the occurrence of an IDLE STACK
condition. During normal operation of a glider, one behavior in the mission file is
controlling glider motors. If one of the required motors is not being driven, an
IDLE STACK condition results.
Types of idle stack conditions include:
MS_ABORT_STACK_IS_IDLE: nothing is commanding the glider.
MS_ABORT_HEADING_IS_IDLE: the fin or roll battery was not
commanded.
MS_ABORT_PITCH_IS_IDLE: the pitch battery was not commanded.
MS_ABORT_BPUMP_IS_IDLE: the buoyancy pump was not
commanded.
The most common cause of a synchronous abort is a device error, which is
generally a device driver returning an error due to some mechanical or electrical
irregularity. The command use performs the following functions:
use <>: displays a list of all devices, whether they are installed and the
number of oddities, warnings and errors associated with that device.
An oddity is exactly that: something odd has happened, but will
NEVER result in the device being taken out of service. A warning is
returned when something fairly serious has happened. If enough
warnings occur, the device will be taken out of service. An error is
either when something very serious has occurred or is the result of a
number of warnings. The device is immediately taken out of service.
use + [device_name] <>: puts device_name in service
use – [device_name] <>: takes device_name out of service
use all <>: puts all installed devices into service
use none <>: takes all installed devices out of service
Two commands can be used to set the number of warnings allowed before a
device is taken out of service (although it’s not recommended to change the
default settings unless you know more about the vehicle than the engineers at
Webb Research). The commands are:
setnumwarning [#] <>: changes the number of warning allowed before a
device is taken out of service while in GliderDOS (ie: no mission is
running).
setdevlimit [device_name] [os] [w/s] [w/m] <>: sets the number of
warnings allowed before a device is taken out of service while a
mission is being run. [os] refers to the number of times the device is
attempted to be put back in service, [w/s] is the number of allowable
warnings per mission segment and [w/m] is the number of warnings
per minute.
For more information on setting device warning limits in mission files, consult
Appendix B.
During the second type of abort, referred to as an out of band abort, the glider
assumes that the software is no longer reliable. For this reason, the upper 2
layers of software control are no longer utilized. Instead, only the device
drivers/schedulers are utilized. Since the higher level software control is not
used, no log or data files are recorded.
There are three causes of an out of band abort:
SOFTWARE IN IMPOSSIBLE PLACE: when the software realizes it is in
a branch of the code that it shouldn’t be.
GLIDER STILL UNDERWATER: self-explanatory.
SYNCHRONOUS ABORT FAILS: the last ditch effort to get the glider
back to the surface with inducing a hardware generated abort.
In an out of band abort, the device drivers will be used to achieve positive
buoyancy, GPS fixes are continuously output and communication devices are
turned on. The abort can only be terminated by user keystroke (CTRL + C),
even though the glider is on the surface. The following dialog is presented:
Want to reset the system? (Y or N)
Want to exit to the operating system?
Make sure you know what you are doing before answering Y
Want to exit to the operating system? (Y or N).
In general, you want to reset the system. When the system is reset, you will be
returned to the GliderDos prompt as long as you have chosen to boot into
GliderDOS upon reset as mentioned earlier. In almost EVERY case, you do not
want to exit the operating system (GliderDOS) to PicoDOS. PicoDOS has no
knowledge of the glider or it’s motors, so exiting GliderDOS while the glider is
deployed is not a good idea.
The third type of abort is a hardware gererated abort. The glider hardware is
capable of autonomously generating an abort sequence and getting the glider to
the surface. There is a watchdog circuit in the hardware with a time constant of
either 2 hours or 16 hours, set by a jumper in the hardware. This watchdog circuit
is referred to as COP_TICKLE (Computer Operating Properly). The idea behind
this circuit is to cause the glider to activate a burn wire, which drops a 470 gram
lead slug out of the glider to achieve positive buoyancy. This method of
achieving positive buoyancy is reserved for a catastrophic software or hardware
failure and is the last resort for retrieving the glider.
RETRIEVING DATA
Data acquired during the course of a mission and stored in a .dbd, .sbd, or .mlg
file can be retrieved from the glider either during surface intervals or at the end of
a mission. When the glider surfaces, it attempts to establish communication with
an onshore computer via RF modem or iridium phone. Once communication is
established, the user will be presented with a number of options:
CTRL + R to resume the mission
CTRL + C to interrupt the mission and keep the glider on the surface
CTRL + E to extend the surface interval by 5 minutes
Enter the following to send all .dbd files:
d dbd <>
Enter the following to send all .sbd files:
d sbd <>
Enter the following to send all .mlg files:
d mlg <>
The above commands are used while the glider is still executing a misison. The
user can also choose to retrieve all data files after the mission is completed. If
the glider is not running a mission, enter the following to retrieve all .dbd, .sbd or
.mlg files, respectively:
sendlog dbd <>
sendlog sbd <>
sendlog mlg <>
Once a data file (dbd, sbd, mlg) has been sent, it is moved from the LOG folder
to the SENTLOGS folder, where the file resides until it is deleted by the user.
There are two ways to transfer a file from SENTLOGS to a shoreside computer.
Unfortunately, PicoDOS and GliderDOS do not recognize the wild character (*)
when transferring files using zmodem. Therefore the user will have to manually
enter in each file name when transferring files from SENTLOGS. This can get
tedious if there are many segments to the mission. The way to get around this is
to copy all of the files you’re interested in from the SENTLOGS folder to the LOG
folder. Switch over to the SENTLOGS directory using:
cd sentlogs <>
Then, copy of the files you’re interested in from this folder to the LOG folder
using the following command and notation:
copy *.* c:\log <>
The above will copy all of the files in the SENTLOGS directory to the LOG
directory. To obtain all files from a particular mission, you must denote the
mission number according the previously discussed 8.3 naming convention. For
example, if you want all files from mission 130, enter the following:
copy 0130*.* <>
This will copy all dbd, sbd and mlg files for mission #130 to the LOG directory.
Once you have copied all of the files to the LOG directory, load GliderDOS and
use the sendlog command along with the appropriate file extension to batch
transfer the files to your computer.
DATA PROCESSING
Once a binary data file (either a .sbd or .dbd) has been transmitted back to shore
via RF (freewave) or satellite comms (Iridium), the file must be renamed and
converted from binary to ascii format. This is accomplished by a series of prebuilt binary executable files written and compiled by Webb Research. The files
are located in the glider module:
rename_dbd_files.exe: this executable renames the binary file from the
mmmmsss.xxx format to a name including the name of the glider, the
year, the julian day, the mission number and the mission segment. The
file type (.dbd or .sbd) is also appended. This new naming convention is
displayed below:
GGGG_YYYY_DDD_MM_SS.XXX
dbd2asc.exe: this executable converts the data from binary format to ascii
format so that it may be read and/or processed. The output files which
result from this execution have the following form:
GGGG_YYYY_DDD_MM_SS_XXX.dat
dba2_orig_matlab.exe: this executable creates a matlab matrix key file
which puts headings on the columns of the converted ascii files (.dat).
These three executable files can be run from a number of ways. The most
straight-forward way is from a DOS command window and performing the
following steps:
1) From the Windows start menu, select RUN and enter:
cmd <>
2) Once the command window has opened, set the working directory to that
which contains the transmitted binary files and the three binary executable
(.exe) files. The following sets the working directory to c:\slocum
cd c:\slocum <>
3) The first step is to rename the files. To rename the binary files, enter the
following:
dir /b *.dbd | rename_dbd_files –s <>
If the files to be renamed are of the type .sbd, replace ‘.dbd’ with ‘.sbd’.
The ‘-s’ at the end of this line outputs the results of the renaming process
to the command window. The new names of the individual files will be
output to the screen. The names listed below are just examples:
ru02-2003-015-5-0.dbd
ru02-2003-015-5-1.dbd
4) The next step is to convert the renamed files from binary to ascii and to
create the .m file used to process the data in Matlab. Enter the following
(using the example filenames from above):
dir /b ru02-2003-015-5-*.dbd | dbd2asc –s | dba2_orig_matlab <>
The use of the * wildcard character will result in the conversion to ascii
and concatenation of all of the segments from mission 5, and creation of
the .m matrix key. The results of these two commands are displayed
below:
ru02_2003_015_5_X_dbd.dat
ru02_2003_015_5_X_dbd.m
Alternatively, each segment may be converted and kept separate by
entering the specific segment to be converted. In this way, mission
segments may be pieced together into one large mission file prior to
processing or indvidual segments may be processed independently in
Matlab.
The screenshot below summarizes steps 1 – 4.
Currently, there are two other ways in which the above process is carried out. I
have written four Matlab scripts that can be run from a Matlab command window
which will perform the renaming and conversion process:
1)
2)
3)
4)
sbdBin.m
sbdBinCC.m
dbdBin.m
dbdBinCC.m
The ‘sbd’ or ‘dbd’ refer to the type of file to be converted and the ‘CC’ denotes
that the script concatenates all files in the directory.
A quick note on the concatenation process….the binary executable files and the
matlab scripts do not know which files you would like to have concatenated. In
other words, if you have files from different missions, but on the same day, and
they are all placed in the working directory, they will all be concatenated and the
resulting file will take the form:
GGGG_YYYY_DDD_XX_X_dbd.dat
GGGG_YYYY_DDD_XX_X_dbd.m
Or, if you have files from different days and they are concatenated, the resulting
file will take the form:
GGGG_YYYY_XXX_XX_X_dbd.dat
GGGG_YYYY_XXX_XX_X_dbd.m
The java based software package that is currently being developed by Chhaya
Mudgal at IMCS also automates the renaming and conversion from binary to
ascii process. As soon as a glider transmits the appropriate data files back to
shore, the files are automatically converted, renamed and then stored in an
archived folder. So, for long term field deployments, it will not be necessary to
perform this process.
Once the files have been transmitted back to shore and renamed and converted
to ascii, the next step is to process the data into something meaningful. At IMCS,
all data processing is performed using Java and Matlab. While the method used
to process data is very specific to the end product desired, a discussion of the
glider sampling grid, data interpolation and other caveats will help to facilitate this
process.
SLOCUM GLIDER SAMPLING GRID
By manipulating the weight to volume ratio, the slocum glider uses changes in
buoyancy to move vertically in the water column. The addition of wings to the
glider provides forward propulsion that results in a maximum forward gliding
speed of approximately 40 cm sec-1. Although the user can define the cycle time
(u_cycle_time(secs)), or time between recorded data samples, all IMCS gliders
are configured to sample once every 2 seconds (1/2 Hz). These factors result in
the following general sampling grid (For the purposes of data discussion, a yo is
defined as one complete descent followed by one complete ascent.):
One ‘yo’
The first point to discuss refers to the x-axis of the preceding plot. The glider
does not calculate a distance traveled when it reports sensor values. The plot
above uses mission time (m_present_secs_into_mission) for the x-axis. The
glider does, however, report the change in distance traveled in both the x and the
y direction. These measurements are reported in Local Mission Coordinates
(LMC) and the units are meters (m). LMC is measured relative to an absolute
zero point, which is set to the position the glider is in when the mission starts.
The positive y-axis to north and the positive x-axis is east.
In order to caluclate an absolute distance traveled, the change in the absolute
values of lmc in the x direction and y direction (m_x_lmc, m_y_lmc,
respectively) from t0 to t1 must be calculated:
m_x_lmct1-t0 = | m_x_lmct1 – m_x_lmct0 |
m_y_lmct1-t0 = | m_y_lmct1 – m_y_lmct0 |
The distance traveled vector can then be calculated using c2 = a2 + b2:
Distance traveled (m) = (m_x_lmct1-t02 + m_y_lmct1-t02)
Summing this distances traveled over each sampling cycle results in the closest
approximation of the distance the glider has traveled with the given data. Using
this calculation and re-plotting the yo pattern produces the following glider path:
This distance calculation has been automated and included in a Matlab script
called gliderCTD.m.
INTERPOLATION OF GLIDER DATA
The Slocum glider’s vertical and horizontal path through the water column create
a rather unique temporal and spatial dataset, one which can be compared to that
of a boat towed system, although differences in the resulting data grid do exist,
and which requires an interpolation approach significantly different from that used
to interpolate stationary, vertically profiled datasets. This discussion is an
attempt to put a method to the madness when interpolating these data files.
There are primarily two parts to this discussion: 1) handling of the vertical and
(more importantly) horizontal scales and 2) interpolation method.
There are numerous Matlab functions which deal with interpolation of 2-D, 3-D
and N-D datasets. Although Matlab proclaims to be very good at interpolating
non-uniformly spaced data using the m-functions meshgrid.m and griddata.m,
my experience with these functions has shown otherwise when dealing with a
typical glider data grid as discussed and shown above. This problem has to do
with the periods of time when the glider is on the surface transmitting data back
to shore and performing water velocity calculations while being pushed by
surface currents. As is evident in almost any glider trajectory plot, this results in
a data grid which contains areas of a highly sampled water column along with
areas of the water column that are sparsley sampled or not sampled at all. The
following example, taken from an actual dataset acquired during a glider transect
on the West Florida Shelf in January of 2003 (Operation Gulfcast), is used to
illustrate some of the problems that may be encountered in interpolation of a
glider dataset.
When dealing with glider temperature transect information, the first step typically
is to eliminate all Nan (Not a Number) data points and calculate the distance the
glider has traveled. It may also be necessary to filter the datastream to get rid of
noise or spurious temperature readings. Once completed, the ‘processed’ data
will contain, among other columns, a distance column, which begins at or around
0 meters traveled to some distance, x. Another crucial column is that of depth,
measured in meters. This column of information differs from the distance
information in that the distance data is continuously increasing, while the depth
column is alternately increasing in depth as the glider rises and then decreasing
as the glider begins to glide to depth. Obviously, the third essential column is
temperature.
Once the data has been filtered, the matlab function meshgrid.m is used to
create a uniformly spaced grid over which the temperature information is to be
interpolated. Although the resulting grid is uniform, the interpolation method
called in griddata.m ensures that all interpolated data passes through the actual
data points acquired during the transect. For help on the syntax used to execute
these two functions, consult the matlab function reference, online help or type:
help [function.m] <>
in the Matlab command window for a description of its usage.
The resulting interpolated pseudocolor plot (pcolor.m) is shown below:
The data points have been superimposed on the image to illustrate the
“smearing” effect, either up or down in the water column, where the data is more
sparse compared to areas which have a relatively dense sample number. It also
appears as if the isobars are being pulled up or down along with the glider.
These artificats are most likely a combination of the differences in frequency of
data points in the x-direction (distance) compared to those in the y-direction
(depth) and the interpolation methods used.
Two methods may be used to interpolate the glider data using griddata.m:
bilinear interpolation or bicubic interpolation. A brief explanation of these
methods and how each relates to the x and y scales reveals the origin of the
errors. For 2-dimensional data, bilinear interpolation fits a bilinear surface
through the existing data points. Interpolated points are a combination of the four
closest real data points. Likewise, bicubic interpolation fits a bicubic surface
through the existing data points and interpolated data points are a combination of
the sixteen closest points, thus producing a smoother surface. For the glider
dataset used in this example, and for any glider dataset covering long distances
and involving multiple surfacing events that result in the glider traveling on the
surface without collecting vertical profiles, the frequency of vertical data points is
much larger than that in the horizontal. Therefore, regardless of whether the
method is bilinear or bicubic, the majority of the 4 points (bilinear) or 16 points
(bicubic) from which an interpolated data point is derived are much more likely to
be taken from the y-axis and much less likely to be taken from the x-axis. This is
especially true when the glider surfaces and, thus, a gap in the dataset is
created. Because of the gap in the data along the x-axis, the interpolation is
performed using more data points in the y-direction compared to the x-direction.
This results in a vertical “smearing” that is evident in the interpolated pseudocolor
plot. While not as dramatic, the same explanation is most likely the cause of the
smearing or dragging of water up and down in areas of high data density.
In order to decrease the discrepancy between the frequency of datapoints in the
vertical and horizontal directions, it has become necessary to trick Matlab into
thinking that the horizontal distribution of data points is much closer to that in the
y-direction. This is done by creating the grid (meshgrid.m) and then dividing the
resulting x-axis values (xi) and the actual distance data points by a constant. In
this example, the constant 0.001 was used. These new values are then used as
inputs to griddata.m. Once the interpolation is complete, the x-axis values and
the real distance data points are multiplied by the constant to bring them back to
their original values. This results in the following pseudocolor plot for the same
dataset used above:
Processing of the CTD temperature data and the CTD salinity data using the
previously described interplation and other processing methods is currently done
in the matlab script gliderCTD.m.