Calibrating MESH via OSTRICH using DDS algorithm
(OSTRICH v1.8 r03.28.2011)
By
Amin Haghnegahdar
University of Waterloo
Civil & Environmental Engineering
ahaghneg@uwaterloo.ca
Adopted and updated from the OSTRICH manual v1.6 by L.S. Matott
for
IP3 Network Final Results Workshop
7-9 September 2011
Introduction

A model-independent calibration and optimization tool

Dr. L. Shawn Matott (2005)
http://www.civil.uwaterloo.ca/lsmatott/Ostrich/OstrichMain.html
http://www.groundwater.buffalo.edu/software/Ostrich/OstrichMain.html

Most important features of OSTRICH is that it is model-independent

OSTRICH uses minimization for optimization/calibration

OSTRICH implements classic numerical methods and popular heuristic algorithms
o Dynamically Dimensioned Search (DDS)
o Particle Swarm Optimization (PSO)
o Genetic Algorithm (real-coded and binary-coded)
o Simulated Annealing (3 variants)
o Gauss-Marquardt-Levenberg (GML) non-linear regression
o Fletcher-Reeves
o Steepest Descent
OSTRICH can calculate numerous regression statistics following a successful calibration exercise
o Observation Residuals
o Confidence Intervals
o Parameter Variance-Covariance
o Influential Observations
o Parameter Sensitivities
OSTRICH is available for both Linux and Windows platforms
o an MPI parallel version of OSTRICH for Linux-based parallel clusters
o OSTRICH can be configured to execute parallel versions of the underlying model executable, if
one is available
o OSTRICH GUI module using Visual Basic
Unconstrained Optimization
o parameters are continuously varying
o The only constraints are upper and lower boundaries (so-called ’side constraints’)
 Can incorporate constraints via penalty functions



1
General Constrained Optimization (GCOP) in OSTRICH

Chapter 6 of Documentation and User’s Guide Version 1.6

Anything other than least-squares calibration or pump-and-treat optimization, needs either
o i) generate a driver program (see Chapter 7) or
o ii) GCOP module

Math and Statistics: Statistical output for GCOP is not meaningful

We use the DDS algorithm (Tolson & Shoemaker, 2007)
o Heuristic stochastic global optimization
o Designed for efficient automatic calibration
o Has only one algorithm parameter (neighborhood size perturbation, r)
o Normally no parameter tuning required, default r=0.2
o Generates good (not best) results in modeller’s time frame
http://www.civil.uwaterloo.ca/btolson/software.htm
Input files (chapter 4 and 6):
 ostIn.txt:
o
Main OSTRICH input file that controls its settings
 Template files:
o
*.tpl files used to replicate the required model input files (explained later)
 Other files:
o
Batch/Script file to execute the simulation model and to supply OSTRICH with necessary files to
calculate cost (objective) function (explained later)
2
Configuring ostIn.txt:
Note: Can use “#” to comment out lines or insert comments
ProgramType:


Choice of the optimization algorithm: e.g. “DDS”, “ParticleSwarm”, etc. (section 4.2)
o Should know the exact name : we use DDS
Or use “ModelEvaluation” for a single/few model runs with the specified parameters and their values in
this file (can be used for quick testing)
ObjectiveFunction:

Always “GCOP” (General Constrained Optimization ) when anything other than least-squares calibration
or pump-and-treat optimization is used
3
ModelExecutable:


Model (or batch file) that will generate the necessary files needed by OSTRICH
Like “OST-MESH.bat” here (details later in “Other files” section)
PreserveBestModel: (not in the older versions or manual)


The batch file that preserves the simulation model (e.g. MESH) input files associated with the best found
parameter set by the optimizer in OSTRICH in the folder named “best”
Like “save_best.bat” here
File Pairs: (See section 4.3)



To pair (generate) the required model input files from their corresponding template (.tpl) files
There should be a template file pair for any input files containing design parameters
General syntax
4


<sep> shows where one filename ends and the next one begins. Semi-colon character ’;’ and the TAB
character. Spaces are not valid separator characters because OSTRICH allows spaces within file names.
Example
BeginParams (section 4.5)


Calibration parameter names, values, bounds and transformations (if any)
General syntax


<name>: The name of the parameters as used in the template file(s)
<initVal> : Initial value of the parameter in units specified by <txIn>
o A value or “random” or “extract” (extract from the existing input files)
<lowBnd> : Lower bound of the parameter, in units specified by <txIn>
<uprBnd> : Upper bound of the parameter, in units specified by <txIn>
o Also check the “minmax_parameters.txt” for allowed ranges by MESH




<txIn>, <txOst>, and <txOut>: These specify the type of transformation to be used by input files,
internally by OSTRICH and in the output files.
o none: no transformation.
o log10 : log base 10 transformation.
o ln : natural logarithm transformation
Example: Note: <format> was added recently to accommodate fixed format used by MESH
5
BeginInitialParams



Sets (lines) of design parameter values in the same order as specified in “BeginParams”
Used only for the ModelEvaluation mode
Example
BeginResponseVars (see sections 6.1 and 4.4)

Response variables that OSTRICH should read from model output files prior to evaluating costs and
constraints

<sep> shows where filename ends. Semi-colon character ’;’ and the TAB character. Spaces are not valid
separator characters because OSTRICH allows spaces within file names.
6

<key>, <line>, and <col>: First, OSTRICH Positions the output file parser at the first line in <file>
containing <key> word. If OSTRICH should begin parsing at the beginning of the file, then <key> should
be set to OST-NULL. Next, the parser uses the <line> and <col> values to locate the position of the
desired observation value. This value is then extracted and converted to a double precision number. The
parsing process is repeated until all observation values are read.

The <line> variable tells OSTRICH how many lines must be skipped, starting from the line containing
<key>, before the line containing the desired observation value is reached. Therefore, if the observation
value is on the same line as <key>, then <line>=0;

The <col> variable tells OSTRICH which column in the line contains the desired value; where each
column is separated by the <tok> variable, and column numbering begins at 1.


The <tok> variable is used to specify an alternative column separator to use when parsing the model
output file; the default column separator is whitespace (any number of space or TAB characters). If model
output uses an alternative format (such as comma separated values), then the <tok> variable should be set
accordingly.
Example


Note: So the value of NS is read by OSTRICH from first column of the 1st line of the “function_out.txt”
function_out.txt is generated by the Nash-Sutcliffe code and looks like
7
BeginTiedRespVars (see also section 4.8)

Response variables whose values are computed by OSTRICH as functions of one or more response variables
and/or parameters


<name>: The name of the new (tied) response variable(s)
<np>: The number of non-tied response variables and/or parameters. Valid values depend on the choice of
<type> (=actual number of variables/parameters when <type>=“wsum”)
<pname1>,<pname2>...<pnamenp> : The list of non-tied response variables and/or parameters


<type>: The type of function relationship between the tied and non-tied response variables. Valid values for
<type> are such as “linear”, “exp”, “wsum”
o wsum: The tied parameter is the weighted sum of the listed parameters.

<type data>: Depending on the choice of <type>, the syntax of this field varies
o If <type> = ”wsum”, the <type data> field should list the values of each weight, using the same
ordering as the preceding named list of parameters.

Example: Turns the regular Nash-Sutcliffe value to the negative NS since OSTRICH uses “minimization” for
optimizing
8
BeginGCOP

Example: NegNS is what is used eventually by OSTRICH to calculate the cost function
9
BeginConstraints

We did not use any constraint here
BeginDDSAlg (not in the Manual)





DDS-specific settings
Parameter: Neighborhood size perturbation, default=0.2
Number of maximum iterations (=objective function evaluation)
How DDS initializes the search (random or specified initial values)
Example:
10
Template files:




OSTRICH uses the template files to create syntactically correct model input files in preparation of
running the model
The contents of the template file should be identical to the paired model input file
o Values of calibration parameters are replaced with unique parameter names defined in the
ostIn.txt file
Be very careful with the names, their lengths and with spaces
Example:
Other files:


Extra (batch) files required to run the model and calculate the objective function (cost) by OSTRICH
Example: OST-MESH.bat
11

1_3_002: the current MESH executable

WtNash_MESH.exe: calculates the weighted Nash-Sutcliffe coefficient for the listed stations based on
the “MESH_output_streamflow.csv” file (located in BASINAVG1 folder)
o Requires the input file “NS_Ex-Stations.txt” in the same directory (BASINAVG1)
o Weights may be assigned based on the average station flows
o Considers optional period for model warm-up (spin-up)
o Use -1 as the station number to calculate weighted NS for all stations
o Example
Single Station (weight does not really matter in this case):
12
Multiple Stations:
13
Running OSTRICH (chapter 8)

To check before running

Place the required files in the working directory (e.g. “ost” here)
o This directory is located in the main directory of the model (e.g. MESH here)

Running OSTRICH
o Run “Ostrich.exe” directly
 No further organizing of files/directories and no progress screen (only for windows)
o Run a customized batch file (e.g. “Ostrich.bat”)
 To create an output folder and transfer output results into it
 To see the progress screen for each simulation (windows only)
 Etc.
 Often used as opposed to Ostrich.exe
o Example:
14
Output files (chapter 9):
 OstOutput0.txt:

Main output file, contains an optimization record along with statistical output (if applicable) and the final
best solution found
15
 OstErrors0.txt:

Error file, any errors encountered by OSTRICH are stored in this file.
 OstModel0.txt:

A sequential record of every model run is stored in this file.
16
 OstExeOut.txt:


The outputs of model runs are redirected to this file.
The output of each new model run will overwrite the contents of this file
17
18