In the current phase of Clean Water Productive Land (CWPL) programme, there are specific goals to:
Add more spatial detail to SPARROW to better identify critical source areas. SPARROW is the software which is used to calibrate parameters for the national/regional contaminant load models contained within CLUES. The initial focus is to add spatial detail to SPARROW, but the modifications should be able to be implemented in CLUES in the future.
Consider whether farm boundaries can be added to improve linkages between farm-scale and catchment-scale models. It is not intended that CLUES will act as a substitute for
OVERSEER or other farm-scale models which require detailed assessment of farm conditions, often supplemented with additional spatial or non-spatial data such as OlsenP levels.
Improve linkages with OVERSEER to enable better representation of losses from farms, for example improved representation of farm practices.
When considering changes to SPARROW, consider implications for the future incorporation of the new erosion/sedimentation model into CLUES.
This document proposes a design for a new SPARROW model to meet the above requirements.
The SPARROW model, which was developed by the USGS, is built around a particular spatial configuration and associated data structure. The basic spatial element in CLUES is the stream reach, and there is one subcatchment associated with each reach. The reaches are in a dendritic drainage network. Each reach has a number of attributes such as:

Source quantities of each source type in the associated subcatchment (e.g. areas of different land uses, erosion terrains, point sources)

Delivery variables such as rainfall and slope for the subcatchment

Other reach attributes such as stream length, topology, and flow rate used in routing of contaminants down the network.
The quantity of contaminant entering the stream from source type j in a particular subcatchment is determined by:
𝑆 𝑗
= 𝑎 𝑗
𝐴 𝑗
∏ 𝑓 𝑘
(𝐷 𝑘
; 𝑏 𝑘
) (1)
1

where A j
is the quantity of source type j in the subcatchment (e.g. t/year from point sources in the catchment, area of urban land, t/year leaching from dairy land), a j
is a coefficient for the source type
(e.g. yield from urban land use, correction factor for point sources or OVERSEER sources), and f k
is a delivery function describing how the source is modified as a function of subcatchment-average delivery variables D k
and associated parameters b k
(the delivery variables are denoted by the subscript k). The
 symbol refers to the product of terms on the right. The same delivery function is used by each source type, except that the delivery term can be switched off for selected j k combinations by setting f to equal 1. The total source from the subcatchment into the associated reach is determined by summing over the different sources. The loads are then routed down the stream network, accounting for losses en route to the catchment outlet.
SPARROW was originally developed in the statistical software SAS, and this has been used in previous NIWA studies. The reach and associated subcatchment data are stored in a simple table
(the reach table, Table 1) with a single record for each. The assignment of column names in the table to variables in the model is established in the header of the SAS code, which can be modified easily by the user. For example, the user can specify the names of the columns that are used as source variables.
Table 1. Reach table records in the current model
Reach
NZReach (REC reach identifier)
Quantity of each source type (A j
)
Delivery variables (average over the subcatchment)
Stream parameters
The coefficients are stored in a separate table of coefficients, and these are associated with the source variables. The coefficients can be calibrated in the model fitting exercise.
In the national NZ nutrient SPARROW models, the source quantities are:
 leaching from pasture, which is pre-calculated from the area of the land use and subcatchment-average parameters such as slope, rainfall, and dominant soil type (see the section on OVERSEER for further explanation)
 leaching from cropping land uses, pre-calculated from a lookup table derived from SPASMO models
 area of various pastoral land uses (to allow additional sources beyond the OVERSEER losses) and the area of various non-pasture land uses. These areas are obtained by overlaying a land use map with subcatchment boundaries (in data preparation for SPARROW, or as a separate modelling step when introducing scenarios for CLUES).
 point sources
 sediment load from Hicks model (for additional background erosional P losses)
The delivery terms are:
 mean annual rainfall
 soil drainage class
2

For the E. Coli model, the only land uses are pasture and non-pasture. For the sediment model, the source terms are the area of each erosion terrain, while the land uses are treated as delivery terms.
Other delivery terms in the sediment model are the subcatchment average slope and mean annual rainfall.
The SAS implementation of SPARROW reports various results for each reach, including the load from each source type entering the stream reach, the attenuation to the catchment outlet, and the load in the reach.
The data structure within CLUES is similar. The OVERSEER calculations are performed in CLUES for each model run. A percentage mitigation of each pastoral source type can be specified for each reach/subcatchment, as can the stocking rate passed to OVERSEER. Only the total source and load for each reach is stored and displayed, rather than a breakdown by source type.
One of the key limitations in the context in terms of increasing spatial resolution is that delivery variables such as slope and soil type are averaged over a subcatchment. For example, in the nutrient models, the same slope is applied to all land uses in the subcatchment, whereas in reality the slope could vary systematically with land use (e.g. dairy on the flats, native vegetation on the hills). While a subcatchment can have a number of different land uses (identified by the areas or proportions of the catchment), specific information on the location of the land use is not used in the model calculations. Similarly, a single dominant soil class is used, whereas in reality there can be more than one soil class, and in reality the contribution to the load from a sub-dominant class (e.g. stony soils or poorly-drained area) could in reality be more important in terms of load generation. In the sediment model, the source variables are the areas of different erosion terrains, but an average slope and average proportion of pasture are used in the calculations. This has the potential to introduce biases and errors in the determination of loads.
The lack of explicit spatial resolution below the subcatchment level also limits the way in which critical source areas can be identified. Currently, critical source areas can be identified at a large scale, for example, by mapping the variation in loading across subcatchments with contrasting land use. However, one might also want to be able to see the effect of different soil types, slopes, land uses, and so on, but the current model does not allow for this (except where the variation in those attributes occurs at a scale coarser than the typical subcatchment scale of ~0.5 km 2 ).
With the current data structure, it would be difficult to allow explicitly for variations at a scale smaller than the subcatchment. Some workarounds are possible but cumbersome. For example, to account for the co-variation of erosion terrain and land use, separate source terms could be retained in the model for each combination of erosion terrain and land use. This would extend the number of columns in the reach table, but this would become unwieldy. For example, using three land uses would approximately triple the size of the reach file, which is already large. Each combination would have to be coded as a separate source, and care would have to be taken to combine source coefficients appropriately. Mapping of the results at the finer scale would be possible if the contribution from each sub-source were reported, but this would involve some new re-mapping steps. For complex combinations of terrains, slope, land use, and perhaps land management, such workarounds would be too cumbersome and complex to be workable. As an additional concern, the
3

SAS implementation of SPARROW for the national models is currently at the limit of data size (2GB array in the IML library) so that increasing the data size would make the SAS version intractable without deep code modifications to break up the large arrays.
The current structure also caused difficulties when specifying mitigation measures. In CLUES, an average mitigation value for each land use within a subcatchment is used. If the user wants to allow for different mitigation in different parts of the catchment (for example, only on one soil type), they must undertake calculations externally to CLUES to establish the average mitigation in the subcatchment. This entails moderately complex data pre-processing which would be too complex for most users. If more spatial detail were enabled, then different mitigations in different parts of the subcatchment could be specified within CLUES in a simpler fashion.
At present, farm boundaries are not taken into account in SPARROW or CLUES. The default land use does incorporate farm boundaries, but when it comes to running SPARROW or CLUES, the land use information is summarised at subcatchment level by determining the areas of land use in each subcatchment. Also, the results are only reported at the subcatchment level, not the farm level.
Moreover, mitigation measures are only input at subcatchment level. Hence, in the current models, the links between farm-level activities and catchment-level outcomes are obscured or lost. This may be of some benefit in cases where there is sensitive farm-level information and there are agreements to hide farm-level information. However, there will be other cases where farm managers and land managers wish to make the link between farms and catchments. For example, for erosion control in the Manawatu, modelled linkages between farm plan implementation and reductions in sediment loading have been important in providing support for plan implementation.
This does not mean that all farm-level information can be captured. For example, erosion control plans are often specified on a spatial scale that would not be practicable to capture in a catchment model. Nevertheless, incorporation of farms as a spatial unit will be a step in the right direction.
A difficulty with the current implementation of SPARROW is that prediction uncertainties cannot be generated. While the SPARROW code allows for prediction uncertainties, the size of the nationalscale models is such that the capabilities of SAS are overwhelmed.
In summary:
Addition of spatial detail would enable more accurate calculations, improved identification of critical source areas, more precise specification of mitigation measures, and inclusion of farm boundaries.
It would be unworkable to add extra spatial detail to the current SAS implementation of
SPARROW, due to limitations in the data structure and size of data.
The key proposed modification is to allow for more than one spatial unit (SU) within each subcatchment. This will require a new data structure and calibration software.
The proposed new data structure is shown in Figure 1. The data for stream reaches are separated from data for the SU’s, so there are two main tables, which are linked through the reach/subcatchment identifier NZReach.
4

The SU’s would be generated by a spatial intersection of various layers (Figure 2). Attributes from the contributing layers, such as land use and soils properties, would be linked into the table of attributes for the SU’s. In addition, averages of some attributes, such as the rainfall, would be determined in GIS and linked to the SU table. The land use layer would represent a combination of land cover and land use information (as in the current LUNZ model) and could also include farm/property boundaries (which delineate changes in land use or land management).
For the purposes of SPARROW calibration, the losses from OVERSEER and SPASMO could be precalculated and incorporated into the SU table. In CLUES, the OVERSEER calculations would most likely be done on the fly. In CLUES it would also be desirable to route each source separately, for use in source attribution calculations.
Each spatial unit would have a number of different land uses, as in the current model. This is to allow for: a) the case where source data provides a mix of land uses on a property without specifying where the land use occurs and b) users specifying changes in land use without having to say exactly where within the spatial unit the land uses occur c) pasture areas where the mixture of pasture types is unknown, in which case regional average proportions pasture types are specified instead.
Currently, the land use areas or proportions are represented with separate columns in a flat file. It is proposed it is proposed to keep the land use information in a linked table, which will make it easier to introduce more land use or land management options in the future, and will allow for more compact data (don’t need to retain a lot of null data in the flat file). However, there are some drawbacks to this approach. Additional coding complexity will be required, and due to the 1-to-may nature of the relation, it is not straightforward to view the land use and associated management attributes in a single table (without performing a very complex query). If this approach becomes too cumbersome, the flat-file option could be used.
The erosion terrains are included for the sediment SPARROW model. For CLUES, the erosion terrains are not retained, but a baseline erosion assuming forested areas is pre-calculated for use in CLUES. If
SedNet is used as the sole erosion model, erosion terrains might not be required, which would reduce the number of polygons overall (but note that in reality the savings would not be great because the erosion terrain boundaries mostly coincide with the soils boundaries because they are derived from the LRI).
It is proposed that slope classes should be used at the overlay stage because they could be important for P and sediment generation. If they are not used, then contrasts in slope would not be captured in the model. To keep the number of polygons manageable, the slope classes should be fairly broad (at most, 10 classes). To get finer precision of the slopes, the average slope within the spatial unit can be determined from summarising a slope raster by spatial unit.
It is also proposed to modify the way in which sources are calculated, to harmonise the approach to different source types. For example, for OVERSEER N losses, all the variables affecting the yield are passed to an OVERSEER dll, rather than being calculated in the form of Equation 1 that is used for other sources. This is workable for Sparrow at present, because the loads from Oversee are precalculated. However, this results in some conceptual clarity and loss of flexibility in CLUES. Hence, within CLUES it is proposed that the sources be considered as a generalised method
𝑆 𝑗
= 𝐴 𝑗 𝑔 𝑗
(𝑫; 𝒃) (2)
5

where D are the variables required for calculating the source and b are the parameters that are calibrated.
Figure 1. Spatial elements and associated records. The dashed line is the subcatchment boundary associated with one reach. Stream reaches are the blue lines. Spatial units are coloured polygons with black boundaries. One farm boundary is shown as a green line.
6

Figure 2. Spatial overlays used to create spatial units.
This proposed approach has implications for SPARROW calibration. As discussed in the previous section, the new data structure and size means that the USGS SAS code cannot be used for calibration. We therefore proposed to transfer the key SPARROW calculations and code into R. We have already tested that this is feasible for the current N model, and the calibration actually runs faster. This allows for different optimisation methods and handling of uncertainty (e.g. bootstrapping).
Within the new R code, source and routing calculations would be done in separate modules. There is currently separate in-line code for these components in our initial R version of sparrow, so there will not be a need for much code refactoring. The key change would be to work with the new data structure, and to summarise the sources by NZREACH. Both of these would be fairly minor modifications.
7

The data will be considerably larger than in the current model. Preliminary analysis (intersecting subcatchments, soils, and LUNZ) suggests that without slope classes, there will be approximately a
10-fold increase in number of spatial elements. Adding slope classes would increase this further by approximately 3-fold. When the slopes of the NIWA 30m elevation raster were classified into 6 classes, there were typically 3 slope classes within each of the subcatchment-LUNZ-soils polygons.
The SU table could then become large (in the order of 10 million elements). Adding erosion terrains would increase this further, except that the boundaries of the terrains are mostly coincident with the boundaries of the soils/LRI. Considering the large file size, it might be necessary to segment the
SU file for the purpose of calculating sources, and release memory once the sources have been calculated. This is not much of a problem, because the sources do not interact and only a relatively small amount of data is required for the routing component.
R can handle linking of tables (for example, linking farm attributes through a farm ID), but coding could be easier and more efficient if large flat files (dataframes) are used rather than a number of linked tables. This decision can await detailed design around the time of implementation.
Calibration of the current N models in R takes a few minutes. The computation time would be expected to scale linearly with the number of elements in the SU file. The sources calculation, which is where the bulk of the extra calculation is required, would also be amenable to simple parallelisation, if this is necessary to achieve faster calculations. Note also that we could use a compute cluster for this calibration work, or a PC with 6-8 cores.
The extra spatial detail has implications for data pre-processing. First, the generation of polygons for the SU’s would require intersection of up to 6 layers, some of which are quite complex. Vector intersection procedures have improved in ArcGIS, to the point where they may be suitable for the large intersections, especially if done on a region-by-region basis (an intersection of subcatchments,
LUNZ and LRI took 1 hour for the North Island). We have previously used raster-based overlay procedures to intersect subcatchments with land use, to speed up the calculations, and this same approach could be used if vector intersections take too long.
The increased data size also has implications for CLUES data and computing time. Currently, CLUES is broken into regions to make the computations and file sizes workable. The load and routing calculations typically take about 3 minutes for a region, and this would increase by an order of magnitude. Hence additional attention to speed of the calculations would be required. We are currently converting CLUES to vb.net, which is likely to improve computational speed. Nevertheless, we may have to use multiple cores and additional code optimisation and transfer of critical steps in faster languages. Overlaying of new land use is also likely to be relatively slow due to the larger number of SU’s, so additional optimisation would be required.
At present, farm boundaries are not taken into account explicitly in SPARROW or CLUES. Even if imported land use layers use farm boundaries from Agribase, the land use is still summarised at subcatchment level. It is worth considering the benefits and difficulties of including farm boundaries.
Benefits:

Can specify farm practices and mitigations on a farm by farm basis.
8


Improved conceptual linkages between farm-scale actions and catchment-scale outcomes, promoting the community adoption of improved practices.

Consistent with the CWPL goals of linking between scales

Results could be displayed on a farm by farm basis, highlighting potential problem areas

Would provide added value to dairy industry efforts to collect information on each farm

Improved potential to capture farm-level losses (e.g. dairy shed effluent disposal) which are logically associated with farm units. Currently the OVERSEER component does not take these into account, which may be part of the reason for the need for additional sources from dairying in the Sparrow models (although there may be other approaches to incorporating farm-scale losses, such as incorporating averages of farm-level losses into the OVERSEER models).
Difficulties

Data size would be increased. This may not be as bad as it seems, because land use is already derived from AgriBase which is on a farm-by-farm level.

National or even regional information on farm practices is not available. Hence, for Sparrow application and model calibration, little use could be made of farm-by-farm information apart from the land use.

Farmers are sensitive about revealing farm information, for commercial, anti-regulatory, or general privacy reasons. Current agreements for Farms Online are unlikely to allow details of farm operations to be made available. However, there will be other cases where catchment collectives, industry bodies, or councils in areas with sensitive catchments wish to make the link between farms and catchments. For example, for erosion control in the Manawatu, modelled linkages between farm plan implementation and reductions in sediment loading have been important in providing support for plan implementation.

There may be unreasonable expectations of accuracy and of ability to link to farm plans. For example, erosion control plans are often specified on a spatial resolution that would not be practicable to capture in a catchment model.

There are some conceptual difficulties linking farm-level information with SU’s. Each farm may contain several SU’s, and each farm may contain more than one land use. If stocking rates are specified for a farm, how should these be spread between different slope classes?
If a mixture of pastoral land uses occurs on a farm, how should they be allocated to SU’s. If there is effluent disposal to land on the farm, which of the SU’s should this be allocated to?
Similarly, which subcatchments should direct deposition losses be allocated to? These issues are considered further in the section on Overseer.
Considering the potential for slow performance of small detailed SU’s, we have considered shifting to a raster basis for the SU’s and calculations.
Advantages of vectors are:
Several base datasets such as land polygons and the LRI/soils are already in polygon form
Easy to link to tables of attributes
easy to understand
9

Can accommodate both small thin elements and large elements (varying spatial detail).
Disadvantages of vectors:
Detailed files can be slow to render.
It is not straightforward to subdivide polygons for segmented geoprocessing
Intersecting can result in slivers (although this is not such a problem as it used to be with older versions of Arc).
Polygons can be bulky if there is a lot of detail
SedNet will be raster based, so some sort of raster component in CLUES will be required ultimately.
Advantages of rasters:
Potentially faster overlays
Tiling, mosaics, and pyramids allow for rapid rendering
Amenable to distributed/parallel processing
File geodatabases in ArcGIS can handle large rasters
Fast open-source libraries are available from geospatial and remote sensing community
Sednet will be raster based
Elevation data has been rasterised already. It is not appropriate to work with contours.
Subcatchment delineation is also based on raster methods, and the subcatchment polygon boundaries are converted from rasters in the first place.
Disadvantages of rasters:
Current CLUES code is based on polygons (apart from the land use overlay).
Need fine grids to capture small features (e.g. thin land parcels) but this same resolution must be used everywhere. However, lossless compression techniques used in may grid formats can overcome this problem. Comparative efficiency of storage compared with vectors depends on the amount of detail in the polygons and resolution of the grid.
Difficulties working with rasters from different datums, alignments, and sizes.
Transformation, realignment and resampling introduce inaccuracies, which can be reduced by decreasing cell size at the expense of increased data size.
Streams and other linear features are usually stored as vectors.
Current Sparrow code is based on records conceptually linked to polygons. It would probably take too long to treat each pixel as a spatial unit. But ‘combine’ raster operations on categorised grids can be used to generate zones with unique combinations of the inputs (e.g slope, land use). The attribute table created in the combine operation could then be used as the basis for the SU attribute table. Sub-models such as OVERSEER would then only need to be calculated for each unique combination (they would take too long on a cell-by-cell basis).
For even greater computational efficiency, the unique combinations could be determined without including the subcatchment boundaries, and the results could then be joined to the relevant SU’s. This introduces some additional steps in processing and remapping results, but reduces the number of times the model needs to be calculated.
If classification/integer rasters are required for computational efficiency, then the precision of variables such as slope will be reduced.
10

May hit processing bottlenecks from older routines in Arc (e.g. in ArcMap 9.3, raster to polygon conversion is limited to 2GB).
Users of ArcMap9 will not be able to use some of the faster/larger raster manipulation features of ArcMap 10.
Conceptually and technically, many of these limitations can be overcome (for example by using unique combinations). Some experimentation should be done to compare the alternatives in terms of overlay efficiency, accuracy, and data storage implications.
Both methods will yield the required SPARROW input, that is, an SU table. Moreover, the geoprocessing in preparation for SPARROW only needs to be done once. Hence the choice of raster versus vectors will really only come into focus at the CLUES implementation stage.
What resolution of raster would be appropriate? An overlay of SC, LUNZ, LRI resulted in approximately 2 million elements, or an average area of approximately 50,000 m 2 , or about 200 m by 200 m. Many of the elements are thin and irregular, however, so a grid of 200 m would not be appropriate. Based on visual examination of the overlay, a scale more like 10-20 m would be appropriate (for example, to capture long thin elements such as riparian bush in the LCDB2 layer or roads in the land use with reasonable accuracy). The slope classification based on DEM’s also results in irregular shapes of polygons, requiring a fairly small grid size.
Current situation
Currently, a cut-down version of OVERSEER is used for SPARROW and CLUES for predicting N and P losses from pasture areas. The OVERSEER calculations are accessed through a dll provided by
AgResearch (Figure 3). The dll is called for each pasture land use in each subcatchment.
Internally within the dll, various default settings have been set up by David Wheeler, which are then used in standard OVERSEER calculations for a single block. If a stocking rate of zero is handed to the dll, then default stocking rates based on the region code are applied. The fertiliser application rate is assessed based on the region code.
Currently, CLUES uses default stocking rates based on the slope and region and passes these to
OVERSEER; these can be modified with stocking rate multipliers provided by the user.
An extension has been provided to take account of stony soils, although this has not yet been implemented within CLUES or Sparrow.
11

Inputs
Pasture type (intensive, deer, dairy, hill)
Stocking rate (cows or SU)
Soil order
Drainage class
Slope
Rainfall
Region code
OVERSEER dll
Generates inputs for
OVERSEER engine via standard farm types. Calls engine for block.
Figure 3. Current use of OVERSEER in CLUES and SPARROW
Outputs
N and P loss per ha
Desired improvements
The current approach has proved easy to implement and it provides for simplicity and rapid calculations. However, there are several areas where improvements could be made:
Include farm-level losses. The losses output by the OVERSEER dll do not take account of farm-level losses such as dairy shed waste or direct deposition to streams. To compensate, additional source terms have been introduced into SPARROW, but they are empirical and it is unclear how to interpret them (for example, should mitigation measures just be applied to the OVERSEER component, or to the OVERSEER plus the additional source).
Make use of the new OVERSEER API (due to be released in January).
All for manipulation of other aspects of OVERSEER such as DCD application, fertiliser timing and amount, pond waste treatment, imported feed, and irrigation method.
Proposed new approach
There are several options to enable the above improvements: a) Allow for complete flexibility. The user can enter all information for each block on the farm and also farm-level information. Probably each block would correspond to a SU. To achieve this, large datasets would have to maintained, as well as writing a user interface to manipulate and format the data. This would be overly cumbersome and essentially unworkable when applied at a national or regional scale. b) Use OVERSEER input files prepared as part of industry or council exercises. This is unworkable in general because the information is not available nationally. This might be an option in the future if more data becomes available. In that case, a switch could be provided to allow the available OVERSEER file to be used. Even so, this would be of limited use for scenario analysis, unless the scenario analysis capabilities of OVERSEER are used. c) Set up a range of OVERSEER files at various land use intensity. Depending on the intensity for the particular SU, the appropriate file is used and fed to the API. The difficulty with this approach is that files would have to be set up for each combination of rain class, slope class, and so on. Also, there is no clear way to manipulate other farm or management variables.
12

d) Use a number of template farms with associated OVERSEER input files, but prepare a utility to modify these based on a limited range of attributes such as stocking rate, rainfall, and key management measures. Then call the OVERSEER engine with the text xml files for each farm.
This approach is likely to be slow, due to delays in reading and writing the xml files. A benefit of this approach is that in principle, OVERSEER inputs could be interrogated by passing the input file to a standard OVERSEER user interface. However, it would not be practicable to do this for every SU/farm, because there would be too much data nationally. Another benefit of this approach is that OVERSEER files from other sources could be substituted if available. e) Wrap the OVERSEER engine with a dll, and use it in a similar way to the current dll, but pass an increased range of parameters and calculate farm-level losses. This has the advantage of added speed (data passed to the engine in memory), and is consistent with the current approach. A disadvantage is that the dll appears as a ‘black box’. This could be overcome by enabling output of the OVERSEER file for selected SU’s. Overall, this last option is the most appealing.
OVERSEER could be used for crops as well. The current SPASMO lookup table is a bit sparse, so it would make sense to use OVERSEER directly.
The key pastoral parameters that we wish to be able to alter are:
Enterprise type (dairy platform, intensive sheep and beef, deer, hill sheep and beef, dairy support)
Stocking rate (but also allowing for default values)
Fraction irrigated
Rainfall
Soil drainage (whether poorly drained)
Soil order
Region
Slope (or topography – note that in OVERSEER these are broad classes)
Fraction with underdrains
Effluent disposal (irrigated, to stream, treatment type)
Stock access to streams
Dairy wintering options (off farm, wintering pad or barn)
Grazed fodder crops
Shallow stony soils
Additional parameters would be required for the fruit and crop components, but for the time being the aim would be to set up the dll to input crop type, rainfall, and maybe soil type.
A key remaining difficulty how OVERSEER farms and blocks relate to the SU’s and subdivision of SU’s into multiple land uses. OVERSEER works with a number of farm pasture blocks that are combined into a farm. In addition to the inputs for each block, there are also farm-level inputs such as stock numbers, housing, and effluent disposal. Losses are output for each of the blocks, plus at the farm level.
On approach to this would be to treat each pastoral landuse within each SU as a block. If stocking rates are specified at the farm level, then the stock could be distributed across the blocks. The
13

OVERSEER dll would have to assemble these into a multi-block OVERSEER file, with additional farmlevel inputs as available from the farm layer. The block-level losses could be applied to assigned to each SU, and the farm-level losses could be distributed to SU on an area-proportional basis. This would involve some intricate coding, both for the OVERSEER dll and for the Clues interface.
An alternative approach is to treat each pastoral land use within each SU as a single-block farm.
Farm-level attributes would be provided to the OVERSEER dll, and the resulting losses would take the farm-level losses into account. This provides a simpler approach which will approximate the full solution. Some distortions could be introduced, for example, effluent in reality might be applied only on some soil types or slopes. But then end result is unlikely to be much different, and in the previous option the farm-level losses had to be distributed across the SU’s anyway. Hence, This second option is preferable.
Key points and ‘straw-man’ decisions from the above are as follows:

Modify the spatial structure to allow for more than one spatial unit (SU) within each subcatchment. The SU’s will be defined from a spatial intersection of several layers, as shown in Figures 1 and 2. Each SU can have a number of different land uses

Separate the stream reach information from the catchment information. Similarly separate source calculations from stream routing calculations.

Plan to keep the land use information as a separate table from the SU table, rather than combining into a large flat file. If this becomes difficult to work with (for coding or communication reasons) then revert to a flat file.

In new coding, attempt to treat Overseer sources and other sources in a unified way conceptually, that is, the source is calculated as a function of various attributes of the spatial unit and land use, rather than separating out the source and delivery terms.

The data will be an order of magnitude larger that previously, so a transition away from SAS is required. It is proposed to use the R statistical software for this task.

The source calculations are amenable to parallelisation, and this should be considered in the design of the R code.

The extra spatial detail might require a shift to raster-based processing. More experimentation is needed before this is done, though. Even with rasters, use would have to be made of unique combinations of inputs with associated attribute tables, so the data form for the core SPARROW calculations would change little.

It is proposed to use farm boundaries as an input layer. Farm-level attributes such as waste treatment type for dairy farms would be associated with this layer. This would involve some difficulties, but there are additional benefits, so the approach should be tried at least as an experimental research exercise.

It is proposed to retain the current method for using OVERSEER, that is, a dll with a fairly limited range of inputs, which would serve as a wrapper for the full model. Potentially,
Overseer xml files could be written for selected spatial elements to improve transparency, but this would be too bulky to use in general.
14


The OVERSEER dll should be extended to accommodate additional inputs to allow for more specific representation of mitigation measures, and also for input of farm-level information such as waste treatment.

Each land use within each SU will be treated as a single-block farm.

The OVERSEER dll should be extended to accommodate fruit and cropping, in place of the
SPASMO lookup table.
15