Processing ADCP raw data to a directional wave spectrum
Status as of 9/10/07
Step 1: The Splitter (python)
The first step is to run a raw output file from the ADCP through the splitter program in
python. This program takes a raw waves and currents interleaved binary file and splits it
into a separated waves and currents binary files. The currents binary file is output in the
same format as an ADCP currents raw file, thus it can then be processed with any other
preexisting code, or with RDI’s software if you so choose. The binary waves file that is
output can also be processed through RDI’s software if needed, however we wish to
process the binary waves output completely independent of RDI. It should also be noted
that the Splitter can handle either a single raw time sample (basically one complete waves
sample with interleaved currents) or a string of samples extending for days or months.
Step 2: Proc Waves or ADCPwaves (python)
Next we take the waves raw file and run it through another python program which
processes the raw data into 4 separate text files ready to be loaded into matlab. These
include a pressure record, a range record for each of 4 beams, a radial velocity record for
each of 4 beams at 5 particular bin depths (for a total of 20 radial velocities), and a
system information file which contains information such as the bin layout, date and time,
heading, pitch and roll, etc. Proc Waves can also handle either one waves time sample
or string of samples. Note that eventually this will be included as an option in the splitter
as part of a module.
Step 3: Specmultiplot (matlab)
This matlab function takes the generated text files from Proc Waves as well as the ascii
version of the RDI Wavesmon waves spectra and outputs polar plots of the directional
wave spectra, 2D plots of the spectra integrated over either frequency or direction, as well
as time series plots of various wave statistics (sig wave height, peak period, direction of
peak period, and dominant direction). The function simply needs to have a directory
where the Proc Waves and Wavesmon text files are located and a start and stop time,
along with a desired time step in hours. Eventually we would like to have Specmultiplot
and its sub-functions all running within python. Note that there are several matlab
functions running within Specmultiplot including the following:
Radial to uvw (matlab)
This function takes out any bad values of the text files and converts the radial
velocities in instrument coordinates into uvw velocities in earth coordinates. It
also uses the heading, pitch and roll to compute the surface locations of each of
the 4 range beams, and the x,y,z positions of each of the radial velocities. The
data is then organized into the structures necessary to be input into DIWASP.
1
These include a structure to compute the spectral matrix with pressure and uvw
velocities, ranges (surface elevation change) or radial velocities.
DIWASP (matlab)
DIWASP (Directional Wave Spectra Toolbox, created by David Johnson) takes
the data inputs and generates a directional waves spectral matrix, a graphical
representation and information about the spectrum (significant wave height, peak
period, direction of peak period and dominant direction). Different spectral
estimation methods can be used, and resolution and iteration options can be
adjusted. The spectral matrix output can be saved and analyzed within matlab and
independently from DIWASP. Note that DIWASP has been edited to improve its
flexibility and the quality of the generated wave spectra. This includes changing
outdated matlab functions (most notably csd to cpsd), adding the ability to use
along beam radial velocities as inputs and the ability to plot multiple spectra on
the same page.
2
Analysis
To establish the best combination of methods for estimating directional wave spectra we
are comparing the results of various methods applied to test cases from an ADCP
installed at Bogue Inlet Pier. The ADCP is a 4 beam unit, with 20 degree beam angles
and a frequency of 1200khz at roughly 8 meters of water depth. We generated spectral
matrices for 2 cases of 4 different relative wave types: long period large amplitude, long
period small amplitude, short period large amplitude and short period small amplitude
waves. For each test case we have generated spectra using the computed uvw velocities
and pressure data, mean elevation change at 4 points along the surface from the range
data, and the radial velocity data. Each of these cases use either the EMEP(Extended
Maximum Entropy Method) estimation method or the IMLM(Iterated Maximum
Likelihood Method) as these are both supported through DIWASP and give the best
results. Wavesmon uses the IMLM method as its default setting. We have also
compared the effect of changing the frequency or directional resolution of the spectrum,
as well as the effect of a new matlab spectral generation function cpsd.
UVW and Pressure Inputs
We first examine directional wave spectrum generated from a combination of the uvw
velocities and the pressure data. The velocities are calculated by using the 4 along beam
radial velocities and computing the uvw velocities along the vertical z-axis in earth
coordinates. This is currently being done at 3 different bin locations, bins 2,3 and 4
which in this case are located at 4.45m, 5.45m and 6.45m from the bottom respectively,
with the depth being roughly 8m. Wavesmon chooses its own bin locations when
processing the data, but at least in these cases, has chosen bins 3,4 and 5. We have
decided to choose 2 over 5 due to bin 5 being very close to the surface and thus at times
can contain numerous bad data values and may suffer from increased distortion from the
surface. This distinction may have a slight effect on the spectrum output, especially if
there is increased inaccuracy at the surface bin chosen by Wavesmon.
Based on the test cases we have analyzed so far it appears that the EMEP method works
very well with the uvw velocity and pressure data. It generally gives a well resolved
depiction of wave energy, without any negative energy anomalies. Figures 1 and 2 depict
this rather well. In both of these polar plots the EMEP uvw method compares well with
the Wavesmon output. However, when the IMLM method is used with the uvw and
pressure inputs the result tends to be poor. Again referencing figures 1 and 2, the IMLM
uvw plot lacks some of the directional constraint that the EMEP method achieves and
contains some energy anomalies which can distort the spectrum. These anomalies tend to
increase when the wave period is shorter (figures 5, 6).
It should also be added that although the EMEP uvw method provides a good spectral
estimate for the two cases of long period waves, when used to estimate the spectrum of
relatively smaller period waves it performs relatively poorly. Referring to figures 5 and
6, the use of uvw and pressure data, regardless of fitting method, performs poorly. It
suggests the method cannot resolve the directionality of the spectrum and thus estimates
3
some energy from nearly every direction. This may be because the velocity data is
attenuated at depth. While the long period waves provide large, consistent orbital
velocity readings well below the sea surface, the orbital velocities of these smaller period
waves perhaps lack strength and consistency at depth. This attenuation is perhaps
combined with the overall increased noise generated by what is probably a choppy sea
(since these are most likely smaller, locally generated wind waves), to cause a decrease in
the signal to noise ratio, especially at depth. This causes the uvw methods to lack the
ability to resolve the wave’s direction effectively and thus provides a rather poor
directional estimate.
Thus it appears that the EMEP uvw and pressure estimation method provides a good
spectral estimate in the case of long period waves, but should not be used for relatively
small period waves. It is uncertain what the effects would be on this method if the ADCP
was placed in deeper water. It is also unclear at this point why the direction of peak
period and dominant direction estimates are so different for the EMEP uvw method and
the Wavesmon output (table 1). Despite this, the significant wave height and peak period
are always very similar.
Range Inputs
The second type of data input used is the range data. This data is the along beam distance
from the ADCP to the surface of the water for each of four beams. The vertical
component of this distance is then calculated and demeaned. This gives us the surface
elevation change at four distinct x,y locations at the surface, enabling the estimation of a
directional wave spectrum. Wavesmon has the capability to use this data as well, but it is
unknown as to when it relies on the ranges to generate its spectra. RDI states that the
range data is merely used as a backup and quality check and that the along beam radial
velocities are instead the desired data selection.
When using the range data, both the EMEP and the IMLM methods appear to do a
reasonable job at estimating the directional spectrum (figures 5 and 6) and they both seem
to be fairly consistent with each other and with the Wavesmon output. The IMLM
method does a slightly better job at resolving the spectral and directional peak (figures 2,
6), as the EMEP method is not as well constrained, especially directionally. It would be
good to see if the resolution of the EMEP method improves or not under different
instrument conditions, most notably depth.
Much like with the uvw inputs, the range data seems to perform better for certain types of
dominant waves. In this case the range data provides a good spectral estimate for
relatively short period (short wavelength) waves (figures 5,6), however if the wavelength
becomes too long (fig 9,10), the directionality of the estimate breaks down. This is most
likely because our test cases are all near Bogue Pier in about 8m of water. This relatively
shallow depth prevents the range measurements at the surface from achieving a large
enough aperture to resolve waves of a relatively large period (thus wavelength).
However, waves less than a certain wavelength relative to the aperture of the range
surface readings tend to be well resolved and provide a good spectral estimate. The polar
4
plots of shorter wavelength waves compare well to the Wavesmon output as do the
significant wave heights and the peak periods. In some cases there seems to be a slight
difference in the significant wave heights (table 1), which suggest the range estimates
may be underestimating wave heights in some instances. The direction of peak period
and the dominant direction have some discrepancies between the EMEP and IMLM range
estimates and the Wavesmon estimates.
As of now it appears that the range data can only be used for short period waves (relative
to the water depth of the ADCP), however in these circumstances it very much
outperforms the uvw and pressure data. In addition as of now the IMLM estimation
method performs very similarly to the EMEP method when using the range data, and
even slightly better in some cases (fig 1,2).
Radial Velocity Inputs
Recently we have also been able to input the along beam radial velocities into the
processing scheme utilizing a transfer function from Hashimoto’s (1997) “Analysis of the
Directional Wave Spectrum from Field Data”. These velocities are from each of the four
beams at 3 different bin depths, right now bins 2,3 and 4. This is the preferred method
used by Wavesmon in its own processing. Despite this commonality there are some
differences between the spectra generated through DIWASP using the radial velocities
and those generated through Wavesmon.
As can be seen in figures 7 and 8 the DIWASP generated spectrum using the radial
velocities seems to follow the Wavesmon output fairly closely when using the EMEP
method. In all eight of the cases looked at so far, however, the IMLM method fails to
generate a quality directional spectrum. The reason for this is currently unknown and is
especially odd since Wavesmon uses this method as its default and generates acceptable
spectra using the radial velocities. There may be an error in the IMLM estimation
function in DIWASP causing this problem or perhaps the radial velocity data output
needs to have additional quality control to eliminate other possible errors or noise not
currently being removed.
When looking at the EMEP method, the overall location of the wave energy and
frequency range seems compares well with the Wavesmon output. Oddly, figure 3 shows
the EMEP radial velocity method lacking directional constraint. This appears to be
anomalous, as this method performs well in the short period example (figure 7) and this
issue does not appear again in the 8 test cases, nor in the comparison data of the ADCP at
the Field Research Facility 8 meter array in Duck, NC. In addition, there is a consistent
difference in the significant wave height when comparing the radial velocity data
spectrum to all others. The heights are consistently less for the estimates using the radial
velocities and there is a difference as large as 30cm in the Long period, Large Amplitude
waves. This can be further clearly seen when looking at the directional spectrum
integrated over frequency (Figures 4,8) where the spectrum is significantly less peaked
and clearly lacking as much energy as the Wavesmon generated spectrum. The reason
for this is unknown. This same issue is not seen at the ADCP located at the 8m array,
5
which is in roughly the same water depth and is also a 1200khz unit. It may simply be
anomalous, or be related to how the ADCP unit at Bogue Pier is functioning, however we
have yet to make a determination.
Number of Iterations and Directional and Frequency Resolution
One of the options DIWASP gives us when generating the wave spectra is deciding the
number of iterations to perform for each estimation method. The default number is set to
100, which is fine for the EMEP method, since the number is actually the number of
iterations before the algorithm ‘relaxes’ the iterative calculation. Since this rarely would
reach 100, the iteration number is relatively insignificant as long as it is set sufficiently
high. For the IMLM method, the iteration number is the number of improvement
corrections carried out at each frequency for the original solution. If set too high, this can
lead to significant negative energy anomalies and an inaccurate spectra. We have
decided on using 3 iterations for the IMLM method, as this number is sufficient enough
to provide improved spectra, but low enough to generally avoid negative energy
anomalies. Wavesmon only uses 1 iteration as its default.
As previously mentioned, DIWASP requires specification of directional or frequency
resolution of the spectrum being generated. The Wavesmon output for the eight cases we
are currently analyzing uses the default settings of a directional resolution of 4 degrees
and frequency resolution of 0.00781250 Hz or 1/128th going from 0 to 1 Hz. As of now
we have chosen for our default values a directional resolution of 2 degrees and a
frequency resolution of .01 Hz going from 0 to .4 Hz. We have also compared the default
resolutions to increased frequency resolution of .005Hz and increased directional
resolution of 1 degree. Changing the resolution can certainly have an impact on the
spectra, however, it is generally relatively minor. The default value of the Wavesmon
directional resolution seems slightly coarse. It is unclear as to why the Wavesmon
frequency spectrum uses frequencies up to 1 Hz. Since the sampling rate of the ADCP is
2Hz, estimating energy up to this high of a frequency seems unreasonable. We have
chosen to use frequencies up to .4Hz (2.5 second waves).
DIWASP Cross Power Spectral Density Function
DIWASP was initially using an outdated Matlab function to estimate the cross power
spectral density called “csd”. Matlab has since updated this function to include some
new parameters, e.g. the windowing function used, and the way that the samples overlap,
and renamed it “cpsd”. We have found that this new matlab function using the default
values provides a slightly better spectrum than the old csd function. The cpsd estimates
seem to be slightly more resolved and have less energy anomalies than in the csd
estimates. Thus, the default setting is now to use cpsd and all of the generated figures
and data use this function unless otherwise noted.
6
Goals and future direction
-The next step will be to compare the independently processed spectra from the 8m array,
to the 8m array spectra generated through our toolbox. This will ideally confirm our
toolbox’s processing and the performance of DIWASP as viable, or point out any
inconsistencies if they exist.
-We would also like to include an analysis of ADCP data from both Gray’s Reef and
Lookout Shoals. This would provide us with data from different depths and types of
locations that could further be used to test our processing toolbox.
-A comparison between a Nortek AWAC and the ADCP located at Bogue Pier is also in
order. We now have about a months worth of data from both of the instruments running
at the same location. The processing for the AWAC data has already been implemented,
however, there has been some inconsistencies with the data structures for this most recent
data set. Once the processing has been corrected, this should provide for another good
test comparison for our toolbox as well as a chance to compare the performances of both
the ADCP and the AWAC under identical conditions, and with the same processing.
-Finally, eventually we would like to implement something like DIWASP in python so
that we do not need to switch from python to matlab. Currently I am not sure what steps
to take to do this, but first we probably need to make sure we have an effective means of
estimating the directional wave spectrum in matlab and then think about replicating this
in python.
Table 1- Wave Test Case Stats
Sig Wave Height
Peak Period
Dir of Peak Period
Dominant Direction
Mar 2, 2007 1359 (LA-LP)
'EMEP UVW'
'IMLM UVW'
'EMEP range'
'IMLM range'
'EMEP radial'
'IMLM radial'
'wavesmon'
2.19
2.19
2.03
2.03
1.84
1.85
2.16
9.0909
9.0909
9.0909
9.0909
9.0909
9.0909
9.1429
6
2
6
14
270
46
355
10
352
8
14
12
46
359
Apr 4, 2007 1400 (LA-SP)
'EMEP UVW'
'IMLM UVW'
'EMEP range'
'IMLM range'
'EMEP radial'
'IMLM radial'
'wavesmon'
1.00
0.98
0.92
0.92
0.86
0.86
1.05
5
5
5
5
5
5
4.92
270
36
28
30
24
346
15
14
38
30
30
22
38
27
7
Fig 1- March 2, 2007, 13:59 (Large Amplitude, Long Period)
Fig 2- March 2, 2007, 13:59 (Large Amplitude, Long Period)
8
Fig 3- March 2, 2007, 13:59 (Large Amplitude, Long Period)
Fig 4- March 2, 2007, 13:59 (Large Amplitude, Long Period)
9
Fig 5- April 4, 2007, 14:00 (Large Amplitude, Short Period)
Fig 6- April 4, 2007, 14:00 (Large Amplitude, Short Period)
10
Fig 7- April 4, 2007, 14:00 (Large Amplitude, Short Period)
Fig 8- April 4, 2007, 14:00 (Large Amplitude, Short Period)
Fig 9- April 21, 2007, 01:02 (Small Amplitude, Long Period)
11
Fig 10- April 21, 2007, 01:02 (Small Amplitude, Long Period)
12